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Abstract 


In  June  1974  a  lAlorkshop  on  the  Attainment  of  Reliable 
Software  jointly  sponsored  by  the  ACM  Special  Interest  Group 
on  Programming  Languages,  the  IEEE  Computer  Society  Technical 
Committee  on  Fault-Tolerant  Computing  and  the  University  of 
Toronto  Computer  Systems  Research  Group  was  held  in  Toronto. 
This  report  contains  summaries  of  the  discussion  sessions  held 
at  the  workshop. 


CSRG  is  an  interdisciplinary  group  formed  to  conduct  research  and 
development  relevant  to  computer  systems  and  their  application.  It 
is  jointly  administered  by  the  Department  of  Electrical  Engineering 
and  the  Department  of  Computer  Science  of  the  University  of  Toronto, 
and  supported  in  part  by  the  National  Research  Council  of  Canada. 
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Preface 


On  17-18  June  1974,  the  Computer  Systems  Research  Group  hosted  a 
Workshoo  on  the  Attainment  of  Reliable  Software  jointly  sponsored 
by  the  ACM  Special  Interest  Group  on  Programming  Languages  (SIGPLAN), 
the  lEEli  Computer  Society  Technical  Committee  on  Fault-Tolerant 
Computing  (FTC)  and  CSRG. 

This  workshop  was  attended  by  about  50  invited  participants,  selected 
for  their  active  research  interest  in  reliable  software  and  methods  for 
achieving  it.  The  workshop  consisted  of  six  general  sessions  and  three 
sets  of  parallel  sessions  in  which  working  papers  and  various  ideas 
relating  to  reliable  software  were  aiscussed. 

This  report  contains  notes  taken  by  recorders  assigned  to  each 
workshop  session.  The  intent  was  tc  provide  a  summary  of  the  major 
ideas,  concepts  and  remarks  presented.  The  reader  is  cautioned  that 
statements  have  not  been  checked  by  those  to  whom  they  are  attributed 
and  hence  any  errors  or  misquotations  are  the  responsibility  of 
the  editor. 

It  is  customary  to  assume,  in  informal  workshops  of  this  nature, 
that  the  participants  expressed  their  views  as  individuals  and  in  no 
sense  as  representatives  of  the  organizations  with  which  they  are 
affi  1  i  ated. 

Since  the  working  papers  presented  at  the  workshop  represented 
transitory  stages  in  the  development  of  ideas,  the  decision  was  made 
not  to  reprint  the  working  papers  in  this  report.  It  is  hoped  that 
the  fully  matured  ideas  will  be  presented  elsewhere.  Where  it  was 
essential  to  maintain  the  thread  of  a  discussion,  material  from 
working  papers  has  been  paraphrased  in  the  notes.  References  to 
papers  (ind  technical  reports  available  in  the  open  literature  have 
been  included  where  appropriate. 
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Session;  Software  Reliability 
Chairman:  Doug  Ross 


Recorder:  Horspool 


The  session  began  with  John  GOODENOUGH  giving  a  presentation  on 
Software  Reliability  and  methods  of  achieving  it.  The  main  points  of 
the  talk  were  as  follows: 

Failure  is  defined  to  be  any  effect  or  behaviour  which  is  unacceptable  when 
occurring  under  permissible  operating  conditions;  where  "unacceptable"  and 
"permissible"  must  be  defined  for  each  system. 

Some  examples  of  failure  effects  a'e: 

1.  Wrong  answers,  with/without  warning. 

2 .  Wo  answers  at  all. 

3.  Degraded  performance. 

4.  Impairment  of  future  correctness. 

5.  Destruction  of  user  data. 

But  note  that  not  all  failures  are  equally  important  and  that  lack  cf 
success  does  not  imply  failure. 

Rel  iabi  1  i  ty  may  be  measured  in  terms  of  ,; 

iMTBF  (Mean  Time  Between  Failures)  •  measures  probability  of  failure 
MTTR  (Mean  Time  To  Repair)  -  measures  impact  of  failure 
Availability  (Proportion  of  time  system  is  available) 

To  increase  reliability,  one  should  reduce  number  of  failures  and/or 
impact  of  f ai 1 ures . 

Reliable  software  is  not  necessarily  coi'rect,  because: 

1.  Correct  software  can  fail  (error  in  specifications). 

2.  Incorrect  software  need  not  fail. 

Robustness  (faul t- tolerance)  is  a  property  of  programs  which  succeed  more 
often,  due  to: 

1.  Compensation  for  input  error  or  component  failure. 

2.  Providing  alternate  approaches. 

3.  Requiring  weaker  conditions  for  successful  operation. 

Fail  Safe  Behaviour  is  behaviour  when  af.sumptions  are  wrong. 

To  Achieve  Reliability,  one  must: 

1 .  Define  failure 

2.  Determine  reliability  requirements 

3.  Use  Assurance  methods  -  tools  for  finding  conditions 

4.  Use  Assessment  methods  -  for  correcting  program  deficiencies  and 
for  certification. 

Basic  Reliability  Issues 

1.  What  effect  or  behaviour  is  prohibited? 

2.  Wnen  can  it  occur? 

3.  How  can  it  be  prevented  or  how  can  its  impact  be  reduced? 

4.  Is  the  analysis  complete? 
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Reasons  for  System  Failure: 

1.  Design  errors. 

2.  Construction  errors. 

3.  Out-of- tolerance  component  performance  (e.g.,  a  file  system 
which  gradually  becomes  disorganized  and  eventually  causes  a 
fai 1 ure) . 

4.  User/Operator  erro 
Reliability  can  be  improved 


Prevention 

Detection 

<  Mitigation  of  Impact  > 
Estimation  of  Frequency 
Estimation  of  Impact 


Discussion  following  talk: 

GAINES  wondered  why  the  costs  of  such  reliability  measures  had  not 
been  mentioned. 

McKEEMAN  commented  that  the  whole  talk  had  been  too  oriented  towards  a 
computer  centre  manager,  whereas  the  economic  costs  of  failure  or  reliability 
measures  should  have  been  shown  from  ttie  user's  viewpoint. 

However,  MITCHELL  felt  that  worrying  about  cost  was  an  irrelevancy. 

It  is  highly  desirable  to  factor  costs  out  of  an  analysis. 

KERNIGHAN  agreed,  claiming  that  most  people  measure  the  wrong  costs  anyway. 

ROBINSON  suggested  that  future  correctness  techniques  would  involve 
broadening  the  class  of  errors  considered. 

But  WOLF  disagreed,  claiming  that  the  correctness  of  many  things  must  be 
taken  for  granted.  A  correctness  proo-^  presumes  that  language  semantics  work 
as  intended,  but  each  machine  instruction  has  only  some  probability  of 
being  executed  correctly.  He  also  gave  an  example  of  a  system  where 
reliability  requirements  were  considerably  different  to  requiring  that  the 
system  be  correct.  A  telephone  switch‘:ng  system,  he  worked  on,  has  stringent 
'up-time'  requirements,  yet  erroneously  breaking  up  to  a  hundred  connections 
was  considered  to  be  only  a  minor  error. 


rs . 
by: 


of 


Design  Errors  by  defensive  design, 
vulnerability  analysis  and  knowing 
characteristic  errors. 

Construction  Errors  by  defensive 
programming,  using  good 
programming  style  in  a 
language  with  good  features. 

Component  Deti oration. 

User/Operator  Errors  by  defensive 
use  and  with  recovery  techniques. y 
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Barbara  LISKOV  next  gave  a  presentation  on  proofs  of  correctness. 

The  main  points  in  her  talk  were: 

Proofs  are  achieved  by 

1.  Identifying  proper  program  structures 

-  which  induce  a  proof  structure. 

2.  Designing  suitable  languages. 

3.  Developing  specification  techniques. 

For  inducing  a  desirable  proof  structure,  the  following  control  structures 
may  be  used: 

1 .  Concatenation . 

2.  Conditional  (Case)  Selection. 

3.  Iteration. 

All  of  these  have  the  'One-in,  One-out'  property. 

Modularization  of  programs  leads  to  an  orderly  program  and  an  orderly  proof 
structure.  For  example: 


do 

B 

do 

C 

A  may  be  proved  correct  with  the  axi oms  that  B  and  C  are  correct  and  with 
rules  of  inference  for  the  control  structures  used  only  within  A.  To 
prove  A  correct,  one  must  prove  the  theorem  that  A  meets  its  specifications. 

In  addition,  the  axioms  that  B  and  C  are  also  correct  must  be  proved. 

Program  specifications  are  very  important  and  must  be  complete  and  short, 
because  the  complexity  of  a  proof  depends  on  the  size  of  the  specification. 

The  one-in,  one-out  property  does  not  lead  to  minimal  length  specifications  because 
of  data  connections  between  modules.  When  two  modules  have  a  data  connection, 
their  specifications  are  longer  and  contain  too  much  information,  implying 
that  their  correctness  proofs  are  unnecessarily  long. 

In  general,  programs  are  not  usually  wel 1 -structured  with  respect  to  data, 
because: 

1.  Scope  of  variables  should  be  as  limited  as  possible 

2.  The  role  played  by  data  should  be  well-defined;  in  particular,  data 
structures  should  precisely  capture  the  meaning  assigned  to  the 
data . 

For  example,  an  integer  set  implemented  as  intset=  record  (lim  :  integer, 

set  :  array  [1 . .20]  of  integer) 

Here,  intset  allows  many  values  which  do  not  represent  integer  sets.  Also 
the  meaningful  operations  for  sets  (Insert,  Remove,  Has)  are  not  captured 
by  intset. 

The  language  should  help  with  wel 1 -structuri ng  of  data.  A  language  under 
development  at  MIT,  called  CLU,  has  a  feature,  the  cl uster,  which  supports 
an  abstract  data  type,  defined  by  operations  on  that  type. 
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A  cluster  for  intset  (above)  combines  the  representation  for  the  set 
with  definitions  for  the  Insert,  Remove,  and  Has  operations.  Outside  the  cluster, 
intset  objects  can  only  be  used  abstractly  (enforced  by  strong  type-checking 
at  compile-time).  There  are  no  free  variables  in  OLD. 

It  is  claimed  that  cluster-like  modules  simplify  proofs.  The 
implementations  of  abstract  types  are  proved  separately  from  their  use. 

For  proving  implementations,  there  is  some  i nvari ant  whi ch  is  assumed  true 
on  entry  and  proved  true  on  exit. 

e.g.,  INTSET(s)  e  s.lim  >  0  &  Vi  ,j  (0<i  ,j<s .  lim  &  s  .set[i  ]=s .  set[j  ]->i=j ) 

For  proving  the  correct  use  of  abstract  types,  one  is  only  concerned  with 
abstract  properties  of  data  and  the  state  space  of  data  is  limited  only 
to  legal  states.  In  addition,  operations  are  the  only  way  to  modiiy  data. 

These  operations  must  preserve  the  invariants. 

Much  work  is  needed  in  knowing  how  to  \/rite  specifications,  but  it  is 
possible  that  they  are  minimal  with  cluster-like  techniques. 

Discussion  following  talk: 

MITCHELL  objected  to  the  example  invariant  for  intset  being  defined 
in  terms  of  the  implementation  for  intset.  An  invariant  used  in  proving 
the  correct  use  of  abstract  data  types  should  be  phrased  only  in  terms  of 
abstract  properties  of  that  type,  i.e.,  what  the  user  sees. 

GRAHAM  remarked  on  a  strong  similarity  between  clusters  and  protection 
systems. 

LISKOV  replied  that  the  resemblance  was  especially  strong  if  one  thought 
of  the  operations  as  access  rights. 

TURSKI  put  forward  his  view  that  data  should  be  considered  as  mappings 
onto  memory,  with  special  morphical  properties.  Such  mappings  would  have 
different  meanings  for  different  programs.  It  was  his  opinion  that  there 
were  as  many  modularizations  for  a  data  base  as  there  were  users. 
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Session:  Structure  and  Hierarchy 
Chairman:  Peter  Neumann 


Recorder:  Holt 


Peter  NEUMANN  introduced  the  theme  of  the  session  as  the  relationship 
be  tween: 

a)  stv^ucture  and  hierarchy,  and 

b)  design  methodology,  languages  for  implementation,  and  system 
organ i zation 

He  cited  Parnas's  listing  of  nine  different  types  of  hierarchies. 
However,  Neumann  emphasized  that  there  are  only  two  essentially  different 
bases  of  hierarchies: 

1)  hierarchies  arising  from  functional  dependencies,  and 

2)  hierarchies  arising  from  refinements. 

It  was  noted  that  error  handling  mechanisms  tend  to  violate  hierarchic 
structures . 

Bill  WULF  gave  a  presentation  in  which  he  warned  against  too  much 
emphasis  o;i  hierarchy  -  as  he  put  it,  “Structure  yes,  hierarchy  no". 

His  warning  was  intended  to  help  avoid  careless  imposition  of  hierarchical 
structure  upon  a  system  which  is  not  inherently  hierarchical. 

As  examples  of  unfortunate  impositions  of  hierarchy,  he  mentioned: 

1)  the  strict  hierarchy  of  names  imposed  by  Algol  60,  and 

2)  the  strict  hierarchy  of  protection  and  allocation  mechanisms 
in  some  operating  systems. 

Wulf  concluded  by  stating  that  language  features  such  as  clusters 
or  monitors  can  help  the  design  process. 

NEUMANN:  One  man's  structure  is  another  man's  stricture. 

Larry  ROBINSON  outlined  some  of  the  goals  and  accomplishments 
of  an  operating  system  design  effort  underway  at  SRI.  The  system  has 
been  designed  with  proof  structures  in  mind.  In  particular,  the  system 
is  designed  as  a  “hierarchy  of  formally  specified  abstract  machines",  such 
that  each  abstract  machine  assumes  only  the  correctness  of  lower  levels. 

The  method  they  are  using  to  specify  their  abstract  machines  is 
similar  to  Parnas  modules.  Each  machine  (module)  has  an  internal  state 
plus  two  classes  of  functions: 

1)  V  functions,  that  inspect  the  state,  and 

2)  0  functions,  that  change  the  state. 
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Robinson  stated  that  they  have  written  and  verified  programs  intended 
to  run  on  their  abstract  machines. 

In  using  this  design  method,  thev'e  arises  the  problem  of  verifying 
that  the  operations  assumed  at  one  level  are  correctly  implemented  at 
lower  levels;  Robinson  called  this  the  "mapping  problem". 

At  this  point,  NEUMANN  quoted  Ted  Glaser  as  saying,  "A  system 
is  modular  if  it  falls  apart". 

Steve  ZILLES  presented  ideas  on  structure  and  hierarchy  in  programming 
languages.  The  central  idea  is  that  of  abstraction,  of  which  he  saw  three 
kinds ; 

1)  functional  (done  by  procedures), 

2)  data  (done  by  clusters),  and 

3)  control  (analogous  to  data). 

A  good  way  of  structuring  is  to  limit  knowledge  to  those  who  need 
to  know. 

There  is  some  need  for  flexibility,  so  that  the  programmer  is  able 
to  meet  efficiency  constraints. 

Alan  BALLARD  described  work  on  proving  part  of  the  SUE  operating 
system  to  be  correct.  He  has  used  a  reasonably  informal  approach  and 
feels  that  we  are  a  long  way  from  formal  correctness  proofs  about  real 
operating  systems. 

In  his  view,  each  level  of  a  system  should  be  characterized  by 
a  simple  set  of  properties.  To  show  that  a  given  level  is  correct, 
it  is  necessary  to  show  that  the  properties  of  that  level  hold,  given  the 
properties  of  lower  levels. 

In  his  work  on  proving  the  SUE  Timer  Manager  correct,  he  found  that 
the  main  difficulties  arose  in  analyzing  the  interactions  of  the  operations 
supported  by  the  manager. 

Mike  MELLIAR-SMITH  presented  work  in  progress  at  the  University  of 
Newcastle-upon-Tyne.  He  mentioned  Peter  Henderson's  Pearl  system,  which 
is  a  language  dependent  system  for  top  down  program  construction.  They 
are  now  working  on  a  comparable  language  independent  system. 

Then  he  described  the  Highly  Reliable  Computing  Systems  Project.  One  of 
the  problems  being  addressed  is  mechanisms  for  recovery  from  program  errors, 
difficulty  arises  from  the  complexity  of  error  conditions  and  consequently 
the  complicated  criteria  for  acceptable  behaviour  of  a  program.  The 
mechanism  they  propose  depends  upon  a  hierarchic  set  of  programs  with 
associatec  "acceptance  tests",  that  determine  if  a  just-completed  program 
behaved  reasonably.  If  an  acceptance  test  is  not  passed,  then  the  state 
of  data  as  it  existed  before  the  program  began  execution  is  re-instated  and  an 
alternate  action  is  taken.  He  stated  that  the  approach  is  somewhat  expensive. 


Session:  Exception  Handling  and  Fault-Tolerant  Software 
Chairmen:  Jim  Gray  and  Mike  Mel  1  i ar-Smi th 


Recorder:  Wortman 


Mike  MELLIAR-SMITH  described  the  Recovery  Block  mechanism  being  developed 
at  the  University  of  Newcastle-upon-Tyne. 

A  recovery  block  looks  like: 

Recovery  block 


Acceptance  Test 
Primary  block 


Alternate  block  * 


[diagramatic  representation 
only] 


A  primary  or  alternate  block  can  contain  or  invoke  further  recovery  blocks, 
thus  allowing  recovery  blocks  to  be  nested  within  each  other. 

The  primary  block  contains  the  statements  normally  used  to  perform  seme 
computation.  The  intention  is  to  place  no  restrictions  on  the  computations 
that  can  be  performed  there. 

The  acceptance  test  is  a  section  of  code  that  determines  if  the  computation 
in  a  block  has  been  performed  successfully. 

If  the  acceptance  test  fails  then  the  alternate  blocks  are  executed  in 
order,  each  being  subjected  to  the  same  acceptance  test,  until  either 
the  action  cf  one  of  the  block  causes  the  acceptance  test  to  succeed,  or 
all  blocks  have  been  tried. 

Alternate  blocks  typically  contain  less  (^fficient  or  less  desirable 
ways  of  performing  the  same  computation,  or,  alternatively,  some  less 
desirable  but  still  acceptable  computation. 

An  important  aspect  of  the  recovery  scheme  is  that  the  execution  of  each 
alternate  block  begins  in  the  same  state  as  the  execution  of  the  primary 
block.  That  is,  all  actions  taken  by  a  block  must  be  undone  if  the 
acceptance  test  fails  for  that  block.  To  date,  only  the  undoing  of 
the  effects  of  assignment  statements  has  been  considered. 

The  principle  undoing  tool  is  a  hardware  device  called  a  "recursive  cache" 
that  records  the  old  values  of  all  variables  assigned  to  in  a  block. 
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Discus si  on : 


Acceptance  tests  are  a  crucial  issue.  It  is  hard  to  construct  rigorous 
tests.  The  tests  may  be  expensive  to  execute. 

Undoing  other  than  simple  assignments  may  pose  combinatorial  space  and 
time  problems.  Several  examples  were  oresented  involving  procedure  calls 
and/or  shared  data  where  recovery  appeared  to  be  very  tricky. 


Jim  MITCHELL  described  a  recovery  mechanism  being  used  in  software 
developed  at  XEROX  Palo  Alto. 

The  basic  mechanism  involves  augmenting  the  run  time  environment  of  each 
procedure  with  additional  information  :hat  allows  hierarchical  error  recovery 

The  construct 

SIGNAL  error  name  (argument-1  i st j 

is  used  to  bail  out  in  case  of  an  error.  Each  procedure  call  can  specify 
explicit  handling  for  error  conditions  signalled  by  the  called  procedure 
by  means  cf  "catch  phrases"  associated  with  the  call. 

A  signal  propagates  through  the  procedrjre  call  hierarchy  until  a  catch 
phrase  for  the  signal  is  found. 

A  catch  phrase  can  handle  an  error  or  merely  note  its  existence  in 
passing  and  send  it  further  up  the  hierarchy. 

It  is  possible  to  send  values  from  a  catch  phrase  to  the  signalling 
procedure  via  "RESUME  (arglist)"  so  that  the  operation  that  failed  may 
be  retried. 

The  signal  mechanism  can  be  used  to  allow  procedures  to  clean  up  the  run 
time  environment  in  an  orderly  way.  For  example 

SIGNAL  the  end  is  coming  ; 

Each  procedure  that  cares  to  can  catch  this  signal  and  tidy  up  its  local  data 
structure.  Irrecoverable  errors  are  signalled  as  "ERROR  errorname  (arglist)" 
unlike  SIGNAL,  it  is  not  possible  to  RESUME  the  procedure  generating  an  ERROR 

Di  scussi on : 

The  procedure  that  issued  a  signal  may  want  to  know  if  its  signal  is 
accepted  and  take  alternate  action  otherwise. 

Specification  of  a  procedure  should  describe  all  errors  that  it  might  raise, 
but  each  call  need  not  be  willing  to  handle  all  errors. 

As  a  corollary,  if  you  call  a  procedure  that  raises  a  signal  that  you 
can't  raise  then  you  must  handle  it. 
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There  may  be  some  signals  that  must  be  raised  so  that  the  outside 
environment  can  keep  track  of  what  is  going  on. 

Example:  logging  of  hard  I/O  errors. 
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Session:  Development  Tools 
ChairpersDn:  Mary  Shaw 


Recorder:  Sevcik 


Much  of  the  session  consisted  of  discussion.  Many  of  the  identifiable 
thoughts  that  emerged  were  the  product  of  contributions  from  several  par¬ 
ticipants.  Each  paragraph  below  is  identified  with  the  primary  contributors. 

SHAW:  After  many  years  of  focusing  on  the  design  of  programming  languages 
and  their  compilers,  it  is  time  to  develop  tools  which  operate  on  the 
boundary  oetween  compilers  and  operating  systems.  Current  instances  of 
such  tools  include  program  editors,  automated  documentation  aids,  and 
component  libraries. 

DESJARDINS:  In  large  real-time  systems,  such  as  some  of  those  developed 
by  NASA,  correctness  proofs  are  infeasible.  However,  tnere  is  a  need  for 
"system  development  system"  to  aid  in  the  coordinated  development  and 
combination  of  components  of  large  systems.  Creation  of  a  total  system 
of  this  type  probably  will  require  at  least  five  years,  but  anticipation 
of  exactly  what  is  required  and  initiation  of  a  plan  for  its  realization 
must  start  immediately. 

HETZEL,  SHAW:  It  is  useful  to  classify  the  kinds  of  tools  that  might 
comprise  a  system  development  system: 

0.  specification  development 

1.  static  analyzers 

2.  dynamic  analyzers 

a.  non-cumulati ve 

b.  cumulative 

3.  sieve  (or  screen) 

4.  code  production  aids 

SITES,  SHAW,  HETZEL:  Types  3  and  4  do  support  reliability  since  they 
aid  effective  maintenance,  and  many  errors  arise  in  r§.“doing  wrong. 

WULF:  While  tools  can  be  classified  as  above,  there  are  many  similarities 
between  coding  and  re-coding.  Everything  from  specification  aids  to  debugging 
and  maintenance  support  systems  should  be  considered  to  be  various  aspects 
of  a  single  problem. 

ROSS:  A  total  system  for  developing  and  maintaining  large  systems  might 
be  viewed  as  follows: 

level  1  -  data  base  and  packaging 

level  2  -  compilers,  loaders,  computer  system  simulator 
level  3  -  overlay  analyzers,  language  pre-processors, 
debugging  aids 

level  4  -  input  editors,  analyzer  programs,  output  editors 

All  the  e'ements  on  levels  two  through  four  both  contribute  to  the 
database  and  build  their  activities  on  information  from  it.  The  database 
is  the  foundation  of  the  entire  system. 
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MILLER,  BOE^M:  A  system  such  as  the  one  indicated  above  is  specialized  and 
expensive.  Would  it  be  cost  effective? 

SHAW,  ROSS;  The  costs  of  maintenance  ard  overruns  are  large  and  growing. 

The  trend  nust  be  stopped.  The  system  indicated  above  might  serve  initially 
as  a  guide  for  development.  We  may  start  level  by  level  making  use  of 
whatever  elements  are  available  at  a  given  time. 

MILLER:  The  size  of  such  a  database  world  be  approximately  a  thousand  times 
that  of  the  programs  being  developed  and  maintained. 

ROSS,  BOEHM:  Understanding  the  program  components  and  their  interactions 
will  permit  the  blow-up  ratio  to  be  much  less  than  one  thousand.  As  with 
generalized  data  base  management  systems,  the  space-overhead  of  maintaining 
information  about  information  is  high,  but  it  is  likely  that  it  will  be 
slowly  reducible. 

DESJARDINS,  ROSS:  Who  will  fund  the  development  of  such  a  system?  Not 
the  manufacturer,  because  the  machine-independence  presumed  in  such  a  system 
is  contrary  to  the  basic  interests  of  a  manufacturer.  Not  a  software 
house,  because  too  much  initial  investment  is  required.  An  aggregation 
of  governmental  users  is  likely  to  have  the  best  chance. 

ROSS,  SHAW,  DESJARDINS:  Such  technology  is  feasible  now  with  proper  financial 
support.  However,  users  have  not  yet  realized  the  long-term  value  of  paying 
the  high  short-term  price. 

HETZEL,  SHAW:  Except  for  very  large  systems  the  scope  of  tools  being 
discussed  is  too  expensive.  For  the  mass  of  users,  we  must  isolate  the 
functions  of  those  tools  which  can  be  most  helpful  (and  visible)  at  low 
cost.  These  might  include  test  data  generators,  output  checkers,  sieves, 
and  text  editors  (other  than  those  standardly  available).  Many  such 
products  already  exist.  However,  their  compatibility,  if  any,  exists  only 
by  chance,  and  their  mutual  coverage  of  most  problems  is  doubtful. 

GRAHAM:  Once  having  convinced  a  potential  funding  organization  that  it 
is  impossible  to  complete  large  software  projects  reliably  within  time 
and  budget  constraints,  how  can  we  possibly  assume  that  we  can  ask  them  to 
provide  massive  funding  for  just  such  a  software  development  project.  It 
is  mandatory  that  we  start  small,  just  as  in  compilers  where  simple  arithmetic 
expression  compilers  were  followed  in  ten  years  by  optimizing  PL/I  compilers. 

BOEHM:  The  piecemeal  approach  is  useful  but  creates  interface  problems. 

For  example,  output  from  one  tool  may  have  to  be  keypunched  before  it  can 
be  used  by  another  tool. 

ROSS,  GRAHAM,  MILLER:  It  is  possible  for  programmers  to  average  14,000 
lines  of  debugged  source  code  per  year.  One  super-programmer  has  been 
observed  to  have  produced  in  a  year  30,000  lines  of  source  (160,000 
machine  instructions)  in  v;hich  the  acceptance  test  revealed  only  ten  bugs. 

In  another  project,  the  cost  of  computer  time  in  developing  a  system  with 
a  PASCAL-ish  PL/I  subset  was  only  $1.50  per  line  of  source  code.  The  tools 
they  used  included  a  data  base  (with  editor)  containing  information  on  modules 
and  interfaces,  and  a  preprocessor  for  the  language  to  encourage  or  enforce 
programmer  discipline. 
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ROSS,  WULF,  SHAW:  It  is  not  necessary  to  use  only  the  cream  of  the  cream 
among  programmers.  However,  by  providing  tools  that  enhance  programmer 
productivity,  we  can  and  should  use  or  ly  the  cream. 

WULF:  By  observing  the  economics  of  agriculture,  we  realize  that  perhaps 
it  would  oe  worthwhile  to  pay  poor  programmers  not  to  code! 

WEISSMAN,  BOEHM,  SHAW:  Can  poor  programmers  be  salvaged?  Appropriate  tools 
may  make  the  "right"  way  the  easy  way,  and  thus  encourage  programmers  to 
work  in  the  rich  part  of  the  solution  space. 

ROSS,  GRAHAM:  It  is  difficult  to  quantify  how  far  a  software  system  is  from 
meeting  an  acceptance  test.  Performance  measurement  in  general  is  not 
typically  done  early  in  a  project  because  the  analysis  is  difficult.  Early 
analysis  should  be  encouraged  by  havir.g  tools  automatically  confront  the  user 
with  perfarmance  analyses  after  each  system  modification. 

DEVANEY,  I'JULF:  Programmers  cannot  make  global  decisions  about  appropriate 
data  stru:tures  for  an  entire  system.  Programming  languages  should  permit 
operations  on  structures  to  be  expressed  without  making  a  commitment  to  a 
parti cula''  representation. 

BOEHM,  WULF,  GRAHAM,  HETZEL,  MILLER:  How  does  hardware  make  it  difficult  to 
produce  giDod  software  development  tools?  Errors  should  be  more  carefully 
isolated  in  time  and  space  (such  as  disk  track  in  which  reading  caused  a 
parity  er'^or,  etc.).  Error  propagation  should  be  controlled  and  a  higher 
proportion  of  errors  should  be  detected.  Interrupt  facilities  should 
provide  better  re-creations  of  what  hes  happened  and  in  what  order.  Much 
more  help  from  hardware  is  feasible  if  we  can  just  express  to  hardware 
designers  what  we  want. 

SHAW:  A  proposal  for  how  program  manipulation  facilities  might  be  unified 
was  presented  by  Shaw.  The  proposed  system  encompasses  expressions 
of  tne  program  as  English  specifications,  formal  specifications, 
source  text,  and  executable  code.  Correspondence  among  the  four  versions 
of  the  program  are  maintained  automatically  a's  much  as  possible 

DESJARDINS,  SHAW,  WULF:  Such  a  system  would  help  avoid  or  at  least  detect 
the  high  proportion  of  errors  which  occur  in  the  manual  transformation  of 
one  program  form  to  another.  Only  some  parts  of  the  system  are  currently 
feasible.  Although  we  would  not  be  able  to  automatically  alter  specifications 
to  reflect  the  effect  of  a  hexadecimal  patch  applied  to  executable  code, 
we  could  at  least  cause  the  specification  to  be  marked  as  out  of  date  due  to 
a  change  to  the  executable  code.  Only  manual  assistance  could  regain  the 
coeherence  of  all  program  versions. 
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Session:  Correctness  Proofs 
Chairman:  Larry  Ragland 


Recorder:  Lazowska 


LONDON  began  by  discussing  the  time  requ' red  for  correctness  proofs. 

Some  surprisingly  consistent  figures  wen^  reported:  100  lines/man-month 
with  the  SRI  interactive  system;  100  lin(s/3  man-months  by  hand,  proving 
microcode;  10  lines  of  LISP/man-day ,  inc‘'uding  coding  and  hand  verification; 
10  lines/man-day  of  Nucleus  with  some  automatic  assistance. 

Next,  the  "burning  issues"  in  verification  were  enumerated: 

1)  Does  hierarchy  pay? 

2)  How  do  we  specify  programs? 

3)  Does  automatic  verification  make  sense? 

4)  How  can  one  incorporate  new  knowledge  into  the  system? 

5)  When  to  simplify? 

6)  Credibility  of  proof. 

7)  Credibility  of  assertions. 

8)  Implicit  assumptions. 

9)  Formal  operations  vs.  practical  computation. 

10)  Handling  asynchronous  processes. 

11)  Handling  big  programs. 

Only  the  las':  issue  was  discussed  in  detail,  motivated  by  BROWN's  question, 
"How  do  I  prove  a  large,  unstructured  Fortran  program?"  The  following 
seemed  to  be  the  essential  points: 

1)  Even  large  programs  can  be  proven  if  the  level  of  complexity  at 
each  level  of  the  hierarchy  is  kept  low.  Some  notion  of  hierarchy 
was  agreed  to  be  essential. 

2)  The  proof  of  a  Fortran  program  requires  proving  things  that  can  be 
taken  for  granted  in  more  disciplined  languages.  KIEBURTZ  cited 
COM.MilN  and  undisciplined  GO  TO's  as  particular  difficulties. 

ZILLES  argued  that  languages  forrr  something  of  a  continuum  in  this 
respect;  CLU's  nice  features  can  be  simulated  by  a  pre-processor 
in  ocher  languages,  to  enforce  the  CLU  conventions. 

3)  KING  argued  that  proofs  don't  fail  because  of  little  mechanical 
things  (e.g.,  non-identical  Common  blocks). 

A  number  of  other  issues  were  briefly  mentioned: 

1)  Does  structured  programming  =  provable  programming?  ZILLES  argued 
that  the  mathematics,  not  the  programming,  provided  the  difficulty. 

2)  Must  proofs  be  long?  RAGLAND  pointed  out  that  the  proof  of  a  theorem 
in  mathematics  is  typically  much  longer  than  the  statement  of  the 
theorem. 

3)  LONDON  was  asked  to  compare  his  mechanical  proof  of  a  Pascal 
implementation  of  the  Sieve  of  Eratosthenes  with  his  manual  proof 
of  the  program  written  in  Alphard.  He  stated  that  "apples  and 
elephants"  could  not  be  compared. 
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Session:  Specification  and  Design 
Chairman:  Bill  Wulf 


Recorder:  Brodie 


The  chairman  introduced  the  session  by  starting  that  "all  of  God's 
children"  need  specifications  that  are  complete,  correct  and  comprehensible. 
However,  the  meanings  of  the  above  terms  vary  with  people  and  application. 

Four  fiv6;-minute  introductions  were  given: 

Ted  LINDEN  argued  for  some  form  of  formal  specification  saying  that  a 
distinction  should  be  made  between  the  abstract  object  and  its  state  (which 
could  be  read  by  the  V-functions  of  Pirnas)  on  one  hand  and  the  actual 
implementation  on  the  other.  The  former  should  not  be  hidden  but  rather 
available  via  a  table  and  a  V-function  as  a  rigorous  specification  of  the 
abstract  state  of  the  model. 

Barry  BOEHM  presented  some  data  from  tests  (conceptual  requirements 
analysis)  of  consistency,  to  show  the  current  state  of  the  art  in  large 
scale  systems  contract  software.  The  tests  v;ere  motivated  by  the  observation 
that  the  design/code  error  ratio  was  64/36  overall  and  45/9  error  residue 
at  the  acceptance  stage.  The  test  found  818  inconsistencies  in  a  system 
of  186  functions  with  967  I/O  entries.  In  summary,  he  proposed  that  phase 
one  of  a  project  be  a  simple  coordinated  consistency  check  to  avoid  those 
errors  usually  caught  at  the  acceptan:e  stage. 

TURSKI  proposed  that  these  first-order  inconsistencies  in  a  small 
system  represent  a  small  fraction  of  those  in  large,  real  systems. 

Larry  ROBINSON  presented  a' simple  example  of  the  proof  techniques  they 
are  currently  using  on  large  programs  in  an  O.S.  development.  They  use 
Parnas'  C-  and  V-functions  and  attempt  to  isolate  information  with  respect 
to  data  structure.  He  presented  an  array  update  giving  the  abstract 
machine,  program  with  assertions  and  <.  proof.  It  was  pointed  out  that  his 
proof  was  mathematically  correct  but  syntactically  incorrect. 

TURSKI  presented  seven  morals  to  argue  for  high  level  specifications 
for  the  supplier  and  end  user  as  opposed  to  internal  specifications: 

1.  Software  reliability  is  a  measure  of  a  user's  confidence  in  his  system. 
Reliability  is  (at  least)  a  binary  relation  in  the  product  space 
software-^-users ,  possibly  a  ternary  one  in  the  space  of  software-<-users+time. 

2.  Software  correctness  is  an  abstract  category.  (Aircraft  are  built 
according  to  a  theory  which  is  mathematically  incorrect).  Software 
correctness  is  neither  sufficient,  nor  even,  perhaps,  a  necessary 
condition  of  reliability. 

3.  A  pragmatic  way  to  check  implementation  against  intentions  is  to  make 
them  evolve  simultaneously,  step  by  step;  otherwise  the  time  factor 
(delays  between  completely  specified  intentions  and  original  needs, 
between  implementation  and  specification)  works  against  the  possibility 
of  making  a  user  satisfied. 
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4.  Fault  tolerance  level  depends  on  application  not  on  software. 

5.  Reliability  and  understandabi 1 i ty  considerations  are  pertinent 
for  different  classes  of  users.  Efficiency  is  orthogonal  to 
reliat'i  lity.  Generality  of  the  basic  core  of  software  enhances 
the  reliability  of  its  extensions,  generality  of  final  software 
package  is  not  to  be  encouraged,  litto  for  adaptability.  If  software 
is  reliable,  it  does  not  require  any  maintenance. 

6.  System  reliability  can  be  reduced  to  software  reliability;  program 
reliability  is  either  a  meaningless  term  (if  the  software  used  consists 
of  mary  interacting  programs)  or  is  identical  with  software  reliability. 

7.  I  was  told  that  in  certain  parts  of  the  world,  a  family  doctor  is  paid 
a  fixed  salary  by  the  family,  but  only  as  long  as  everybody  in  the 
family  is  feeling  OK;  he  is  not  paid  any  fees  for  his  services  when  a 
member  of  the  family  is  sick;  he  may  strike  out  a  family  from  his 
register  only  when  all  members  of  ;his  family  are  in  good  health,  or 
all  deceased. 

Turski  argued  that  specifications  and  implementation  must  evolve 
simultaneously.  The  user  can't  be  expt^cted  to  know  the  complete 
specifications  initially.  Further,  he  pointed  out  the  lack  of  work 
in  this  area  for  large  systems  as  opposed  to  that  for  toy  problems. 

WULF  opened  the  discussion,  noting  the  two  points  of  view: 

1.  complete  formal  specifications  before  coding; 

2.  simultaneous  evolution  of  specification  and  implementation. 

An  argument  was  made  for  a  system  of  incremental  proofs  in  which  changes 
would  be  isolated  and  their  effects  easily  traceable. 

ROBINSON  noted  that  high  level  specifications  are  too  susceptible 
to  change  and  that  formal  specifications  offer  a  starting  point. 

SHAW,  arguing  for  both  user  and  detailed  specifications,  made  the  analogy: 

geometry  proof  software  specs 

1  diagram  -  wha :  it  is  good  for  -  user 

2.  proof  -  why  it  is  true  -  implementation 

ROBINSON  and  SHAW  argued  that  Parnas  doesn't  provide  (1). 

SHAW  and  WEISSMAN  want  (2)  with  one  clear  development  path  which 
gives  design  decisions.  HOWDEN  called  the  diagram  an  example  of  use,  not 
a  description. 

Arguments  were  made  for  a  system  \n’th  fixed  and  evolving  specifications. 
WULF  argued  against  the  computational  ndependence  of  what  the  user'  sees  and 
what  the  system  does  but  agreed  that  some  distinction  would  help  make 
some  high  level  specifications  less  susceptible  to  change.  KING  suggested 
that  well-known  aspects  could  be  fixed  but  unknown  (research)  aspects 
must  evolve. 
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ROSS  proposed  a  set  of  specification  types  and  their  corresponding 
rules  which  must  interact. 


Requi rements 
Functional 


User 


Commi ssi oner 
Buyer 
Manager 
doer 


Formal 
Desi gn 
Detai 1 


ZILLES  proposed  that  two  levels  of  specifications  could  exist,  formal 
specifications  for  implementors  and  user-oriented  specifications  using 
English.  Zilles  and  CARTER  emphasized  that  these  two  levels  must  be 
consi ster t . 

GOOEENOUGH  proposed  the  following  additions  to  specifications: 

1.  why  design  decisions  were  made  and  the  extent  of  their  effect; 

2.  characterize  decisions  with  respect  to  likelihood  of  change; 

3.  characterize  decisions  with  respect  to  ease  of  change. 

LINDEN  emphasized  (1)  and  proposed  some  mechanisms  to  maintain  the 
limited  influence  of  decisions:  modularity,  clusters,  successive  refinement. 

BOEHM  re-enforced  1  and  3  above  and  proposed  that: 

1.  specifications  drive  a  performance  model  as  a  test,  then  that  they 
drive  user  documentation  consistently; 

2.  only  critical  areas  be  fully  specified  and  proven. 

The  analogy  between  building  software  and  building  bridges  and  buildings 
was  made:  HETZEL  said  that  engineering  specifications  are  usually  frozen 
before  construction.  He  added  that  perhaps  software  should  be  frozen  in 
smaller  sections. 

WULF  pointed  out  the  1000  year  practical  advantage  in  architecture. 

SHAW  agreed  that  this  advantage  existed  for  knowledge  of  components  but 
not  how  the  components  fit  together. 

TURSKI  disliked  the  analogy  since  building  users  are  interested 
in  what  and  how;  system  users  should  only  be  concerned  with  what. 

HORNING  noted  that  in  talking  to  an  architect  he  found  both  the 
issues  and  the  jargon  of  the  past  twenty  years  seemed  very  similar  in 
both  software  engineering  and  architecture. 

A  discussion  arose  concerning  what  should  or  should  not  be  hidden 
from  user  and/or  implementors. 

WEISSMAN  argued  against  Parnas'  isolation  saying  that  information 
should  not  be  hidden  so  the  programmer  could  relate  better  to  the  whole 
system.  He  added  that  good  programmers  know  what  shouldn't  be  used. 

ROBINSON  disagreed,  since  most  programmers  are  not  good,  KING  supported 
Weissman,  adding  that  isolation  often  results  in  redundancy.  TURSKI  disagreed 
with  Freeman's  suggestion  that  users  were  concerned  with  structure  due 
to  its  implication  on  cost. 

WULF  proposed  that  users'  knowledge  of  structure  allowed  them  to 
get  around  it. 
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HORNING  summarized  with  "It  ain't  what  you  know  that  don't  hurt  you; 
it's  what  you  know  that  ain't  so". 

LEVITT  emphasized  the  need  for  a  (high  level)  powerful  specification 
or  assertion  language.  There  was  little  response  to  his  call  for 
features  or  mechanisms  of  such  a  language.  HORNING  cautioned  people 
not  to  forget  the  past  twenty  years  of  programming  languages  in  designing 
such  a  language. 

Throughout  the  session  people  referred  to  the  problem  of  evolving 
specifications.  Proponents  of  evolving  specifications,  in  one  form  or 
other,  were:  Turski ,  King,  Linden,  Boehm,  Miller,  Shaw  and  Wulf.  Robinson 
was  the  strong  dissenting  voice  although  many  participants  qualified  their 
position  (e.g.,  Wulf  proposed  extreme  caution  in  adding  features.  Gaines 
proposed  the  need  for  a  measure  of  feedback  necessary). 

The  main  topics  of  the  session  were: 

1.  the  need  for  various  levels  ef  specifications; 

2.  evolving  vs.  fixed  specifications; 

3.  the  isolation  of  programmers  or  users; 

4.  the  need  for  a  specification/assertion  language; 

5.  the  necessity  and  type  of  development  paths  in  documentation. 
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Session:  Language  Design  for  Reliability 
Chairmari:  Jim  Horning 


Recorder:  Wortman 


HORNING  presented  some  ideas  on  language  design  for  reliability. 

"Reliability  is  not  an  add-on  feature"  -  B.  Randell 

There  is  no  reason  to  suppose  tiiat  languages  designed  without 
explicit  concern  for  reliability  will  be  suited  for  the  production  of 
reliable  software. 


A  dichotomy  on  approaches  to  design: 


Full  Language  Design 

Size  and  Masterabi li ty 
(restriction  of  power) 


Language  Feature  Design 


Harmful  Constructs 

(discourage  cleverness) 


Error  detection  and  diagnosis 

Human  (readability  over  v;ri tabi  1  i ty) 
Lexical  (mandatory  decla'^ation) 
Syntacti c 
Static  Semantic 
Run-time 

Early  binding 

(correctness  a  compile-time  property) 


Characteristic  error  analysis 
(after  Ichbiah) 

Development  of  useful  structures 

Abstraction  mechanisms 

Checkable  Redundancy 

(types,  ranees,  assertions, 
invariants) 


Explicit  recording  of  decisions 
Mocularization  and  Interface  decisions 


Association  of  test  cases  with 
module 


Features  for  error  recovery 
Visual  structure  (paragraphing) 


Horning  then  enumerated  research  topics  relevant  to  the  study  of 
reliability  in  programming  languages. 

Modules  (classes,  clusters,  capsules) 

Representational  and  Functional  Abstraction 
Separation  of  Naming  -  Specification  -  Implementation 
Axiomatic  Specification  of  Objects  and  Operations 
Structured  Specification 

Declarative  Redundancy 

Strong  Typing 
Invariants 

Other  embedded  properties  (e.g.,  dimensional  analysis) 
Scope  restriction 
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Language  features 

Disciplined  control  and  data  structures 

Characteristic  error  analysis 

Restriction  of  dangerous  operations  (e.g.,  assignment) 

Empirical  validation,  robust  program  structures  (e.g.,  recovery  blocks) 

GANNON  described  research  he  is  conducting  into  the  relationship 
between  programmer  errors  and  programming  language  features.  His  work 
involves  an  empirical  study  of  the  errors  made  by  two  groups  of  students 
using  two  languages  designed  to  support  parallel  processing  and 


producer/ consumer  system  modelling. 
TOPPS 

express! on-oriented 

semicolon  as  a  separator 

automatic  inheritance  of 
environment 

expression  evaluation: 
ri ght-to-left 
equal  precedence 
among  operators 

infix  operators:  &  and  | 

selection:  i_f 

repetition:  repeat 

constants:  literal 

statement  brackets: 
end  and  ) 


TOPPSII 

statement- oriented 

semicolon  as  a  terminator 

inheritance  of  environment 
only  upon  specific  request 

expression  evaluation: 
left-to-ri ght 
"traditional"  operator 
precedence 

functions:  all  and  any 

selection:  i f  and  case 

repetition:  repeat  and  for  each 
(array  element) 

constants:  literal  and  declared 
statement  brackets:  end 


Gannon's  general  hypothesis  is  that  the  language  design  choices  made 
for  TOPPSII  will  lead  to  significantly  lower  programmer  error  rates. 

He  presented  some  preliminary  data  that  supported  this  conclusion,  and 
indicated  that  evidence  concerning  each  particular  decision  will  also 
be  assessed. 

KERNIGHAN  observed  that  the  main  problem  in  most  languages  (read  PL/I) 
was  too  many  features.  75-90%  of  the  features  should  be  removed.  He  felt 
readability  to  be  the  key  factor  in  determining  the  value  of  language  features. 

WEISSMAN  discussed  his  research  on  methods  of  investigating  the 
psychological  complexities  of  computer  programs.  [A  detailed  description  of 
this  work  is  now  available  in  Weissman  1974,  Ed.] 
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ZILLES  discussed  reliability  aspects  of  the  design  of  CLU.  He 
feels  that  the  most  important  language  attribute  is  comprehensibility  and 
that  there  are  several  tools  for  achieving  it: 

-  Abstraction 

-  Simplicity  (e.g.,  the  difference  between  PL/I  and  Algol  semicolon  rules) 

-  Uniformity  (observe  the  difficulty  of  remembering  exceptions) 

-  Syntax  should  reflect  semantics 

-  Eliminate  needless  opportunities  for  error  (e.g.,  for  loop  vs.  do  while) 

-  Know  your  audience  -  choose  primitives  on  the  basis  of  expected  use. 

-  Avoid  unnecessary  synonyms  -  introduce  standard  names  for  builtin 
aspects  of  the  language. 

WULF  discussed  types  of  abstractions  and  differentiated  between 
implicit  abstractions  (those  built  into  a  language)  and  explicit 
abstractions.  His  current  preference  is  to  avoid  implicit  abstractions 
but  to  provide  tools  for  building  thi^m  using  explicit  abstractions. 

Wulf  continued  with  a  description  of  Alphard,  a  new  system  implementation 
language  he  is  designing.  The  basic  objectives  in  the  design  of  Alphard  are: 

-  Type  definitions  similar  to,  and  for  the  same  reasons  as,  CLU 

-  Hide  representational  issues 

-  ‘Proof  criterion 

-  Ease  of  modification  and  maintenance 

-  Retain  abstractions 

-  Efficiency 

The  major  abstraction  mechanism  in  Alphard  is  the  "form".  It 
contains  "everything"  relevant  to  the  realization  of  the  abstraction: 

1 .  data  structures  used 

2.  access  functions 

3.  operations 

4.  initialization  (meaning  of  extent) 

5.  protection 

6.  sequencing  definition  (generators) 

7.  synchronization  (path  expressions) 
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8.  assertions  (invariants) 

9.  exception  handling 


Brief  example  : 


Form  definition; 


Form  set  (p-j ,  P2,  . . . ,  p^) 


formal  parameters 


{  } 

form  body 


Form  instantiation 


decl  x,y: 

declared 
vari ables 


set(ai)  ^25  •••> 

^  ^ 

form  instantiation 

name  parameters 


During  the  discussion  that  followed,  McKEEMAN  attacked  the  whole 
concept  of  'a"  programming  language.  He  felt  that  using  a  single  language 
leads  necessarily  to  complexity.  The  real  problem  is  to  structure  a 
human  activity  called  programming  from  design  to  maintenance.  The  path 
from  conceptual  understanding  to  mechanizable  form  involves  a  whole  catalog 
of  mental  gymnastics.  It  seems  clear  tnat  "a"  language  that  allows  one 
to  worry  about  sequencing  and  data  structures  simultaneously  is  making 
the  whole  process  more  difficult.  In  some  sense  i^lcKeeman  is  askina  for 
intermediate  languages  in  the  region  of  human  translatabi lity  (i.e., 
above  the  mechanization  level). 

References : 

Weissman,  L.,  A  Methodology  for  Studying  the  Psychological  Complexity  of 
Computer  Programs,  Technical  Report  CSRG-37,  Computer  Systems  Research 
Group,  University  of  Toronto,  August  1974. 

McKeeman,  W.M.,  On  Preventing  Programming  Languages  from  Interfering  with 

Programming,  C.E.P.  Report,  Information  Sciences,  University  of  California, 
Santa  Cruz  (to  appear). 
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Session:  Testing,  Validation  and  Verification  of  Software 


Chairman:  J.  Miller 


Recorder:  Lazowska 


This  session  consisted  of  some  introductory  remarks,  a  number  of 
short  presentations,  and  a  general  discussion.  An  attempt  will  be  made 
to  summarize  each  of  these. 

Scattered  throughout  the  session  were  attempts  to  define  the 
terms  in  the  title.  MILLER  argued  that  validation  is  the  comparison 
to  a  standard  with  respect  to  some  property.  Generally,  the  standard 
is  the  program  specification  and  the  property  is  correctness.  LONDON 
felt  that  correctness  is  an  emotionally  loaded  term,  and  that  verification 
is  a  less-highly-charged  alternative.  HOLT  contrasted  verification 
and  validation  by  claiming  that  the  latter  involves  testing,  while 
the  former  does  not. 

MILLER  proposed  the  following  measures  of  program  "goodness": 

correctness 
robustness 
cl  ari ty 
performance 
cost 

portabi 1 i ty 
human  factors 
maintai nabi 1 i ty 
on-scheduleness 


He  claimed  that  while  there  may  be  tradeoffs  between  all  of 
low-level  programmers  are  generally  concerned  only  with  the 
last.  He  felt  that  each  of  the  factors  should  be  weighted, 
of  program  testing  was  characterized  as  "The  program  is  not 
work . " 


these  measures, 
first  and  the 
The  result 
known  not  to 


LEVITT  described  a  machine-implementable  program  verification  method, 
at  least  partially  incorporated  into  an  SRI  system.  He  claimed  that  his  formal 
validation  system  avoids  the  following: 

run-time  analysis 

determination  of  path  conditions 

generating  data  to  force  each  path 

path  proving 

Floyd  verification 

Floyd  assertion  generation 

automatic  synthesis 

LONDON'S  approach  includes  interaction  as  an  integral  part  of  the 
verification  process.  Programs  are  written  in  Pascal;  the  system  includes 
a  verification  condition  generator,  a  simplification  box,  a  theorem  prover, 
and  human  component.  London  equates  verification  with  establishing 
consistency  (i.e.,  between  program  and  specifications). 


-23- 


MILLFR  described  an  automatic  test-case  generation  system  for 
relativelj'  large  Fortran  programs.  The  system  selects  likely  paths, 
generates  inequalities  to  force  the  paths,  then  solves  the  inequalities. 

He  claims  that  the  automatic  system  can  achieve  roughly  90%  path  coverage. 

KING  introduced  his  symbolic  execution  system,  EFFIGY.  The  system 
generates  an  execution  tree  based  upon  the  program  path  conditions, 
requesting  assistance  at  nodes  where  the  path  condition  provides  in¬ 
sufficient  information.  King  also  gave  a  general  overview  of  symbolic 
executi on . 

SITES  discussed  the  system  in  his  thesis.  [Sites  74] 

He  claimed  that  while  correctness  conditions  cannot  easily  be  specified, 
"cleanliness"  conditions  can  be.  His  system  mechanically  generates 
assertions  and  attempts  to  prove  them,  generally  hunting  for  degenerate 
cases.  It  returns  a  binary  decision:  either  the  program  will  terminate 
cleanly  or  it  won't.  Sites  views  this  system  as  a  debugging  aid  rather 
than  a  correctness  box.  He  points  out  that  even  a  certified  algorithm 
may  fail  in  certain  environments;  it  is  this  aspect  of  correctness  that 
he  i s  concerned  wi th . 

There  was  considerable  discussion  centered  around  Dijkstra's 
"Testing  can  only  demonstrate  the  presence  of  bugs,  never  their  absence." 
Among  the  points  made  were: 

GAINES,  LONDON:  There  is  a  large  class  of  bugs  stemming  from  a 
mis-understanding  of  the  environment.  Testing  should  be  used  to  see 
how  closely  the  assertions  correspond  to  the  real  world. 

McKEEMAN:  It's  a  fine  idea  to  pay  for  the  discovery  of  bugs,  but 
if  you  wart  to  stay  solvent,  pay  a  fixed  price  to  the  guy  who  finds  the 
most  bugs  in  a  specified  time  interval,  rather  than  a  fixed  price  for 
each  bug  found. 

In  general,  it  was  felt  that  verification  and  testing  were  complementary 
approaches . 

References : 

Sites,  R.L.,  Some  Thoughts  on  Proving  Clean  Termination  of  Programs 

Techriical  Report  STAN-CS-74-41 7 ,  Computer  Science  Dept.,  Stanford 
University,  1974. 

Sites,  R.L.,  Proving  that  Computer  Programs  Terminate  Cleanly, 

Technical  Report  STAN-CS-74-41 8,  Computer  Science  Dept.,  Stanford 
University,  1974. 
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Session:  Software  Production  Facilities 
Chairman:  Peter  Freeman 


Recorder:  Khaiat 


The  small  turnout  was  due  to  the  similarity  of  this  session  to  the  previous 
day's  "Soitware  Development  Tools"  session. 

FREEMAN  asked  for  a  discussion  of  the  current  state  of  the  art  (tools, 
procedures  and  problems)  in  software  production  with  respect  to  reliability. 

DICKMAN,  augmented  by  Freeman's  questions,  described  the  following  tools  and 
problems  in  his  information  systems  shop:  A  four  step  management  procedure: 

(1)  a  trouble/improvement  report;  (2)  the  action  report;  (3)  the  decision; 
and  only  then  (4)  the  action  design,  change  etc.). 

He  then  described  four  automated  tools: 

1.  a  text  editor  with  source  plus  all  changes  on  files; 

2.  CLEAR  (of  CLEAR-CASTOR)  which  keeps  track  of  changes,  produces 
reports  and  generates  JCL.  Its  problems  include:  can't  add  reports, 
not  integrated  with  management  or  trouble  data  bases,  technical  problems, 
built-in  preconceptions,  oriented  to  specific  languages,  didn't  aid  bug 
isolation  or  current  state  description; 

3.  a  system  to  graph  dependencies; 

4.  a  trouble/correction  report  system  which  was  a  data  management  system 
which  operated  in  isolation; 

Dickman  concluded  that  these  tools  would  make  an  impact  on  reliability  if: 

1.  they  themselves  were  reliable; 

2.  they  interact  (in  a  consistent  way)  via  a  complete  program 
data  base; 

plus  3.  a  semantic  checker; 

4.  a  context  editor; 

5.  a  change  checker. 

BOEHM  and  DICKMAN  would  like  an  automated  system  which  accepts  source  code 
and  produces  a  measure  of  goodness  which  tells  where  more  detailed  analysis 
is  required. 

BOEHM  described  a  TRW  study  of  the  nature  (quality,  benefit,  feasibility,  ease, 
etc.)  of  several  metrics  (core  initialized?,  checkpoint  needed?  etc.).  One 
part  of  the  study  is  the  module  consistency  checker  (see  Specification  and 
Design  session)  which  has  been  used  successfully  in  one  production  situation. 
Boehm  gave  the  following  list  of  tools  (divided  into  system  phases)  giving 
their  stage  of  development  and  anticipated  difficulty  (most  are  simple  but 
labori  ous) . 

1.  Functional  Specifications 

requirements  -  property  matrix:  completed  a  manual  system 
modelling  and  simulation 

formal  review/walk-through:  manual,  brain  storming  in  groups  of  3  or  4, 
best  aid  so  far 
protocol  and  scenario 
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2.  System  Specs,  and  Detailed  Design  (all  of  1  plus) 

trace  tool:  near  completion 
I/O  description  analysis 
proof  techniques:  very  hard 

3.  Coding  Process 

programming  standards:  complet'.ng  a  manual 
language  features 
unit  generation  bids 

library  and  configuration  control:  manual,  one  man 

4.  Code  Analyzer 

str jcture/test  case  selection 
singularity  analysis,  e.g.,  zero  divide 
cross  reference  generator 
source-code  comparison 

(standards)  compliance  checking:  audi tor-fi xer  automated 
test  data  generation:  very  hard 

5.  Execution  Analyzer 

test  data  management  system 
performance  monitoring 
trace 

output  comparison  and  exception  reporting 

interpreter  simulation 

test  skeletons  (top  down  trees) 

Boehm  feels  that  these  tools  have  already  had  a  good  impact  on  reliability. 
A  problem  is  to  distinguish  language/machine  dependent  tools. 

DESJARDINS  suggested  saving  all  text  editor  output  to  record  decisions. 

He  also  proposed  a  'cost  plus  penalty'  contract  which  Boehm  favoured. 

It  was  pointed  out  that  two  major  projects  (now  defunct)  have  attempted 
to  compare  software  and  hardware  production. 

Concl usi ons 

1.  few  new  tools  (perhaps  code  scanners); 

2.  study  in  this  area  leads  to  good  programming  practice  (if  not  tools); 

3.  each  line  of  code  must  be  examined; 

4.  there  is  a  great  need  for  information  on  the  need  for  tools  and  the 
effects  of  tools. 

Reference:  Characteristics  of  Software  Quality,  TRW  Systems  Group, 

Redondo  Beach,  California,  U.S.A. 
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Session:  Machine-Aided  Testing 
Chairman:  Richard  Fairley 


Recorder:  Barnard 


The  following  list  of  issues  was  presented  by  Fairley. 

-  static  vs.  dynamic  analysis 

-  generalized  systems  vs.  specific  tools 

-  relation  to  documentation  and  s  leci fi cations 

-  batch  vs.  interactive 

-  path  analysis  and  test  data  generation 

-  preprocessors,  compilers,  interpreters 

-  sccpe  of  systems  -  types  and  sizes  of  programs  being  tested 

-  goals  of  testing 

-  relation  of  testing  to  verification 

There  was  some  discussion  of  the  semantic  difference  in  the  words 
"correctness"  and  "reliability". 

The  next  issue  was  the  goals  of  testing.  Coverage  of  paths,  or 
execution  of  every  line  was  suggested  and  discussed.  Isolated  data  points  were 
presented  that  indicated: 

-  50%  of  existing  errors  can  be  detected  in  this  way 

-  a  small  number  of  runs  is  necessary  to  accomplish  level  zero, 
or  "every  statement  executed"  coverage 

-  in  some  cases  nothing  significantly  more  is  learned  than  from 
relatively  random  testing 

but  these  cannot  be  used  to  prove  general  conclusions.  The  general 
conclusior  was  that  this  approach  is  useful  in  that  it  may  detect  errors, 
but  is  no  guarantee  that  a  certain  percentage  of  errors  have  been 
detected.  There  was  some  feeling  that  this  is  better  done  by  hand  than 
automatically,  as  the  very  process  of  developing  test  cases  induces  a 
deeper  understanding  of  the  program  and  a  detection  of  errors. 

The  question 

"What  is  the  payoff  relative  to  investment  in  testing  at  various  levels?" 
was  asked,  but  not  really  answered. 

Discussion  moved  to  "generalized  systems  versus  specific  tools". 

What  kind  of  tools  should  we  have  for  machine-aided  testing? 

Suggestions  were: 

-  time  profile  generators 

-  output  comparators 

-  database  initializers. 

It  was  pointed  out  that  mimicing  user  behaviour  is  not  at  all  sufficient, 
as  a  large  user  community  is  going  to  provide  more  cases  than  the  producer 
can  set  up.  An  approach  is  to  exhaustively  test  small  pieces  of  code  so  that 
we  can  have  confidence  in  it  before  inserting  it  in  the  containing  production 
envi ronment. 

"The  best  development  tool  is  a  good  programmer." 
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Session:  Data  Types  and  Reliability  I 
Chairman:  Steve  Zilles 


Recorder:  Horspool 


An  observation  was  made  by  MITCHELL  that  modules  (or  clusters  or  forms) 
were  semant'.c  extensions  to  Simula  67  classes.  This  instigated  the  construction 
of  a  list  of  potential  differences  between  classes  and  modules.  The  final 
version  of  :his  list  was:  - 


1 . 

2. 

3. 

4. 

5. 

6. 

7. 

8. 

9. 

10. 
11  . 


Visibility  Restrictions.  CE.g.»  components  of  a  class  are  accessible 
outsi de  the  cl  ass . ) 

Control  of  Free  Variables.  (Normal  scope  rules  apply  in  Simula.) 
Creation;  static/dynamic  and/or  limitations  on  creation. 

Destruction;  deletion/retention,  locality  of  construction,  zonality. 
Ty[)e  Generation. 

Run-time  Overhead. 

Use" in  multiple  classes. 

Asymmetry  of  Operands.  (E.g.,  being  forced  to  write  commutative, 
binary  operations  as  A.plus(B).) 

Local  variables  have  static  allocation,  but  classes  have  dynamic 
al locati on . 

Declaration  of  types  at  run-time. 

Uniform  Referents. 


During  the  formation  of  this  list,  there  was  much  argument  over  what 
a  data  type  is.  Three  viewpoints  manifested  themselves: 

1.  Types  are  sets  of  objects.  KIEBURTZ. 

2.  Types  are  algebras  consisting  of  sets  and  operations.  ZILLES. 

3.  Types  correspond  to  address  mappings  onto  memory.  TURSKI. 

Stacks  were  in  considerable  use  as  an  example  in  various  arguments. 

They  were  used  by  HORNING  as  a  counter-example  to  an  assertion  oy 
Liskov  that  types  always  have  an  equality  operation.  The  assignment 
operation  was  also  deemed  unnecessary  for  stacks  by  TURSKI. 

Later,  TURKSI  argued  that  testing  stack  equality  is  very  much  a 
dynamic  ope-^ation  when  defined  in  terms  of  popping  elements  from  both 
stacks  and  comparing  them  individually. 

HORNING  countered  with  the  example  of  integers  defined  by  Peano 
axioms,  where  equality  has  a  similar  'dynamic'  definition. 

Much  argument  centered  around  the  exact  difference  between  a  data 
type  and  the  implementation  of  a  data  type.  TURSKI  could  only  think  of 
a  stack  as  a  rather  primitive  addressing  mechanism.  However,  ROSS  had 
most  people's  agreement  with  the  assertion  that  a  "prescription  for  the 
essence  of  'stackedness'"  is  a  stack  type  and  not  to  be  confused  with  any 
particular  implementations  for  stacks.  MITCHELL  put  this  more  clearly 
by  stressing  the  difference  between  an  "abstraction  module"  to  describe 
the  behaviour  of  stacks,  and  a  cluster,  which  is  a  specific  realization  of 
the  abstraction.  KIEBURTZ  contended  that  the  existence  of  pointers  into 
a  stack  did  not  stop  it  from  being  a  stcck,  but  he  found  little  support 
for  this  "free  interpretation"  view. 
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On  the  subject  of  uniform  referents,  WOLF  observed  that  Pascal 
violates  the  rule  by  having  different  selection  mechanisms  for  record 
and  arrc.y  elements/fields .  This  subject  brought  out  MITCHELL'S  claim 
that  no  existing  language  adequately  allows  sparse  arrays  to  have  the 
same  (syntactic)  referencing  mechani  im  as  ordinary  arrays.  ROSS  took 
up  the  challenge  on  behalf  of  AED,  stating  that  the  assignment,  A(i)=B, 
is  compiled  as  a  function  call,  A(i,3).  The  function  A  would  be  called 
with  one  argument  for  evaluation  of  an  element  or  called  with  two  arguments 
for  an  assignment.  Hence,  on  assignment,  A  could  create  or  delete 
elements  as  desired.  However,  MITCHELL  won  the  battle  with  an  example 
of  a  sparse  array  of  record  structures.  ROSS  was  forced  to  admit  that 
A(i).X  =  B  does  not  get  compiled  into  a  form  suitable  for  sparse  arrays. 

[This  discussion  was  continued  in  Session  9b,  page  39.  Ed.] 
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Session:  Operating  Systems 
Chairman:  Ted  Linden 


Recorder:  Cla'^k 


Theme  proposed  by  the  Chairman  -  How  do  we  design  reliable  operating  systems? 


Peter  NEUMANN  initiated  the  session  with  a  summary  of  a  general  design 
methodology  for  hierarchically-structured  systems  which  permits  formal 
correctness  proofs  of  assertions  about  system  security.  Details  on  the 
methodology  are  available  in  [Neumann  74]. 

Briefly,  the  following  five  stages  in  the  design  of  a  complete  system 
were  presented. 


Stage  1 :  Structured  decomposition  of  system  into  abstract  machines. 
-  concept  of  level  embodies  these  two  notions: 

(a)  A  level  is  functionally  dependent  on  (is  implemented 
by)  lower  levels. 

(b)  A  level's  correctness  depends  on  the  correctness  of 
lower  levels. 


Stage  2:  Formally  specify  each  operation  of  each  abstract  machine  (in 
terms  of  input  and  output  assertions  for  each  operation). 

-  uses  a  formal  speci fi cati cn  language. 

Stage  3:  Formally  interrelate  the  specifications  of  each  abstract  machine. 

-  uses  assertions  as  a  specification  device. 

-  at  this  stage,  there  exists  a  complete  design  for  the  system. 

HOLT:  What  is  a  complete  design? 


NEUMANN:  A  complete  design  does  contain  some  implementation 

details,  but  it  is  important  to  separate  specifications 
from  implementation. 

Stage  4:  "Abstractly"  implement  each  operation  of  a  non-primitive  abstract 
machine  in  terms  of  operations  available  on  lower-level  abstract 
machines . 

-  this  can  be  done  in  a  programming  language. 

Stage  5:  Move  software  onto  hardware. 

-  this  may  involve  compilation  (possibly  by  hand). 


Larry  ROBINSON  discussed  the  proof  methodology  corresponding  to 
the  above  design  methodology.  The  major  goal  is  to  produce  a  secure 
system  by 

(a)  finding  a  formal  way  of  stating  what  security  means,  and 

(b)  guaranteeing  this  security. 
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The  above  design  methodology  provides  a  medium  in  which  security 
assertions  could  be  proven.  The  methodology  relies  on  axiomatic  properties 
of  capabilities  to  allow  proof  of  protection  aspects. 


KERNIGHAN: 

Can  you  solve  the  loop  problem?  NEUMANN:  Yes. 

KERNIGHAN: 

Does  this  system  exist? 

NEUMANN: 

Yes,  but  not  on  an  abstract  machine  provided  by  hardware. 

GAINES: 

Proofs  apply  to  abstract  algorithms  on  abstract  machines. 
Reliability  applies  to  real  programs  on  real  machines. 

We  still  need  to  concentrate  on  the  latter. 

NEUMANN: 

A  system  depends  heavily  on  ensuring  that  lowest  levels  are 
the  most  reliable. 

GAINES: 

How  do  you  do  this? 

NEUMANN: 

It's  an  implementation  detail.  We  have  tried  to  separate 
implementation  from  design. 

ROBINSON: 

Yes,  we've  tried  to  characterize  the  behaviour  of  an 
operating  system  in  a  formal  way. 

GAINES: 

Proofs  of  correctness  of  security  assertions  in  terms  of 
the  complete  design  (Stages  1-3)  and  abstract  programs 
(stage  4)  looks  very  hard,  but  possibly  doable.  Proof 
of  correctness  of  implementations  of  abstract  programs 
(compiler)  and  hardware  looks  too  difficult. 

NEUMANN: 

You  can  prove  the  system  is  secure  on  the  basis  of  the  third 
design  stage.  The  rest  is  implementation. 

BREDT: 

Proofs  of  correctness  depend  on  assumptions  about  the 
environment  (e.g.,  bits  of  machine  words)  which  are  very 
hard  to  state,  or  even  consider  explicitly,  let  alone  verify. 

NEUMANN: 

In  that  sense,  the  security  problem  is  intrinsically  unsolvable 

LINDEN: 

How  do  we  get  reliability? 

ROBINSON- 

In  terms  of  design. 

MELLLAR-oMITH :  Do  you  think  that  the  highest  degree  of  redundancy  is 


NEUMANN: 

required  at  the  lowest  level? 

No,  the  highest  level  of  correctness  is  required  at  the 
lowest  level.  Precision  at  the  lowest  level  does  not 
necessarily  require  redundancy. 
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Tom  BRLDT  gave  a  presentation  motivated  by  the  observation  that 
compilers  seemed  easier  to  produce  than  systems  because  specification 
techniques  for  compilers  are  better.  He  combined  the  observation  of 
Wirth  (private  communication)  that  "an  operating  system  is  a  collection 
of  translators"  with  the  observation  of  Brinch  Hansen  that  operating 
systems  involve  the  passing  of  messages  among  processes.  These  ideas 
motivated  the  notion  of  Syntax-directed  operating  system  design,  as 
a  way  of  stating  how  each  process  in  an  operating  system  interprets 
messages  sent  to  it.  [Bredt  72]. 

Specifically,  he  used  syntax  diagrams  (similar  to  those  used 
by  Wirth  in  Acta  Informatica  1971  to  describe  programming  languages) 
to  specify  operating  systems. 

example  -  an  operating  system  portion  which  was  a  collection  of  processes 
mirroring  tiie  hardware  devices  for  I/O.  The  interrupt  handler  is  a 
process  which  creates  messages  for  device  processes.  The  "syntax  diagrams" 
describe  (specify)  how  messages  can  be  passed  among  these  processes. 

Labels  on  connecting  lines  in  the  charts  indicate  semantic  actions  to  be 
performed.  Bredt  pointed  out  that  this  system  is  not  hierarchical. 

ROBINSON:  But  the  processes  all  rely  on  a  lower-level  message  manager. 

BREDT:  That's  right. 

Another  example,  a  disk  driver,  was  presented. 

ROBINSON:  Isn't  error  recovery  a  lower  level? 

HOLT:  Only  if  it  takes  you  back  to  the  same  level. 

NEUMANN:  This  (syntax  charts)  is  poor  For  reliability  if  error 

recovery  is  not  specified. 

General  Agreement. 

HOLT:  A  real  strength  of  this  approach  is  the  possibility  of  a  class 

of  simple  XPL  -  like  systems. 

BREDT:  Maybe  we  could  get  a  class  of  machine-independent  operating 

systems . 

HOLT:  It's  probably  too  much  to  hope  that  this  will  be  able  to  be 

table  driven. 

GOOD:  Raglan  makes  use  of  charts  like  these  to  improve  the  implementabi 1 i ty 

of  the  systems. 

Another  example,  the  ANSI  standard  for  data  communication,  was  described. 

HOLT:  Not  only  do  these  charts  define  the  input,  but  also  the  output 

to  the  next  phase.  They  also  define  complete  specs,  for  error 
recovery. 
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BREDT:  Parnas-like  specs,  work  well  if  the  environment  acts  on  a 

module  (e.g.,  symbol  tables),  but  not  if  the  module  acts  on 
the  environment  (e.g.,  parsers). 

KERNIGHAN  gave  a  user-oriented  description  of  the  UNIX  system 
running  on  a  PDP/11  at  Bell  Labs.  The  theme  was  as  follows:  the 
generality  of  a  system,  as  it  enchances  the  consistency  of  component 
interfaces,  can  increase  reliability  of  both  the  system  and  user  programs. 

Comments  made  by  Kernighan  on  the  system  include: 

-  efficient  interactive  system,  running  on  a  small  machine 

-  constructed  by  two  very  good,  very  experienced  people 

-  written  in  a  BCPL  -  like  language  called  C 

-  incorporates  simplifications  cf  Multics  -  like  facilities 

-  file  system  is  very  general 

-  The  system  was  constructed  incrementally,  with  the  earlier 
parts  used  to  assist  the  development  of  later  parts. 

HOLT  presented  a  piano-player  mcdel  of  process  behaviour.  A  piano 
player  is  viewed  as  a  process  who  plays  a  tune  (an  operation  prescription) 
on  a  piano  (a  data-oriented  object)  ty  successively  pressing  one  key 
(operatiiDn)  on  the  piano  at  a  time.  No  chords  (multiple  notes  played 
at  the  same  time)  can  be  played  on  the  piano,  but  mutually  exclusive 
access  among  several  players  to  keys  on  the  piano  is  permitted. 

The  piano  is  initiated  by  an  external  start  button  which  "creates" 
an  instance  of  it.  The  piano  contains  local  strings  and  hammers  (data) 
and  local  "tunes"  (procedures)  which  implement  the  external ly-visible 
keys  (operations).  The  notes  in  the  local  tunes  may  themselves  be  played 
on  local  sub-pianos  by  local  sub-players. 

Two  conceptual  views  of  playing  the  piano  are  possible.  One  view  is 
that  the  striking  of  a  key  blocks  the  player  until  the  piano  produces 
its  sound.  Another  view  is  that  striking  a  key  gives  the  player  access 
to  the  inner  mechanism  of  the  piano  where  he  can  play  the  appropriate  note 
himself.  In  effect,  he  temporarily  assumes  the  roles  of  the  sub-players 
inside  the  piano. 
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Session:  Review  of  Parallel  Sessions 

Chairman:  Jim  King  Recorder:  Sevcik 

[This  session  consisted  of  summaries  of  the  proceeding  parallel  session,  Ed.] 


Correctness  Proofs  Session  (Ragland) 

The  main  concern  was  with  verifying  the  correctness  of  large,  real 
programs.  The  relationship  between  the  length  of  a  program  and  the  length 
of  its  correctness  proof  was  discussed.  It  was  believed  that  proof  length 
grows  approximately  linearly  with  or  as  the  square  of  the  length  of  the 
program.  Certain  language  features  (such  as  FORTRAN'S  COMMON  and  EQUIVALENCE) 
prohibit  effective  verification. 

Run-Time  Error  Handling  (Gray) 

Mel  1 iar-Smi th  described  a  scheme  involving  acceptance  tests  at  the 
end  of  each  block  and  alternate  blocks  to  be  executed  in  the  event  that 
the  acceptance  test  fails. 

Mitchell  and  Goodenough  proposed  an  alternative  means  of  dealing  with 
return  codes  intended  to  save  both  execution  time  and  space.  The  primitive 
operations  included  SIGNAL,  EXIT,  NOTIFY,  and  RESUME. 

ROSS:  Classes  of  errors  more  general  than  a  particular  error  but  more 

specific  than  "any"  error  would  be  helpful  in  the  proposed  scheme. 
WULF,  MELLIAR-SMITH :  Acceptance  tests  at  a  block  end  must  proceed  any 
real-time  irreversible  action. 

MITCHELL:  The  approach  mentioned  above  differs  from  PL/I  ON-condi tions 

in  that  the  permissible  operations  are  more  reasonably  defined 
with  fewer  restrictions  on  what  could  be  done  where.  The  single 
prohibition  is  that  once  an  error-handling  block  is  activated, 
another  error  of  the  same  type  cannot  be  handled.  Empirically, 
errors  nearly  never  happen.  Therefore  execution  time  devoted 
to  preparing  for  errors  (such  as  PL/I  ON  condition  statement 
executions)  should  be  minimized. 

MELLIAR-SMITH:  While  we  can  be  sure  that  we  record  a  lot  of  information  to 
pinpoint  each  detected  erro”,  we  cannot  know  the  frequency  or 
gravity  of  undetected  errors. 

Development  Tools  (Shaw) 

Topics  discussed  were: 

1.  Management  Tools  (programmer  discipline  and  software  programming  support 
tools  are  both  helpful. 

2.  A  total  unified  system  of  software  development  tools  is  needed  to 
support  design,  implementation,  and  maintenance  of  large  systems. 

3.  A  complete  set  of  compatible  tools  are  needed  (including  design  support 
tools,  clerical/library  aids,  instrumentation  tools,  coding  aios, 
analysis  tools,  test  case  generators,  etc.) 

4.  A  complete  data  base  describing  simultaneously  the  several  program 
forms  (English  specifications,  formal  specifications,  source  text, 
executable  code)  is  necessary  during  program  development  and  for 
maintenance . 
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5.  Such  tools  or  systems  of  tools  will  be  expensive  to  develop.  End 

users  must  be  educated  to  provide  financial  support  for  the  development 
of  such  tools . 


BROWN : 

ROSS; 

GRAHAM: 


and 


HOLT: 

SHAW: 

HETZEL: 


We  must  learn  from  the  end  user,  not  educate  him. 

End  users  don't  know  what  they  want  or  what  is  good  for  them. 

We  must 

1)  convince  the  users  that  they  need  some  large  system  (like  MULTICS), 

2)  convince  them  that  the  system  can't  be  built  without  development  tools, 

3)  convince  them  that  the  development  tools  can't  be  realized  without 
their  financial  support. 

Then  the  users  will  pay  for  the  creation  of  the  development  tools 
which  they  are  convinced  are  necessary  to  build  the  system  that 
they  are  convinced  they  need. 

Fancy  tools  aren't  needed.  Good  compilers,  pencils,  and  erasers 
suffi  ce . 

Having  only  good  compilers  works  to  a  point,  but  not  for  assembling 
and  maintaining  large  systems. 

The  session  didn't  even  address  the  question  of  whether  or  not 
tools  are  needed. 


Software  Production  Facilities  (Freeman) 

Dickman  and  Boehm  described  existing  tools  and  the  extent  of  their 
use.  Many  companies  have  found  development  tools  useful  and  are  creating 
more.  There  is  a  need  for  accurate  measurements  of  the  cost  effectiveness 
in  creating  and  using  development  tools  for  software  production. 

Software  Tools  for  Machine-Aided  Testing  (Fairley) 

In  attempting  to  demonstrate  that  a  program  meets  its  specifications, 
software  testing  aids  may  assure  that: 

1)  every  statement  is  tested, 

2)  every  branch  at  every  decision  point  is  tested, 
or  3)  every  path  through  the  program  is  tested. 

Automatic  test  data  generation  for  exercising  all  paths  is  extremely 
difficult.  It  is  currently  more  reasonable  to  just  provide  program 
examination  aids  so  that  the  programmer  can  more  easily  generate  effective 
test  cases  manually. 

Author  testing  was  cons tras ted  with  validation  or  acceptance  testing. 

While  total  support  systems  are  too  ambitious  for  serious  consideration 
now,  such  tools  as  usage  monitors,  output  comparators  and  data  base 
initializers  are  currently  being  used. 

What  are  appropriate  aids  for  effective  test  case  selection  by  the 
programmer?  Is  there  a  middle  ground  between  just  making  sure  that  each 
statement  is  executed  once  and  complete,  proof  of  correctness  or  exhaustive 
test  generation? 

GAINES,  MITCHELL:  Perhaps  test  cases  should  be  selected  before  implementation 
starts.  Knowing  what  aids  for  testing  are  available  might  guide 
the  design  to  assure  that  it  is  testable. 

LONDON:  Some  people  have  advocated  devoting  up  to  one-half  of  total  system 
development  time  to  the  creation  of  debugging  tools. 
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Data  Types  (Zilles) 

The  relationship  of  data  types  to  reliability  was  not  analyzed. 

It  waf,  considered  whether  whether  data  types  should  be  viewed 
as  sets  or  algebras,  or  whether  data  are  not  types. 

Weaknesses  of  SIMULA  67  classes  were  enumerated. 

MITCHELL:  The  "Zilles  list"  [Page  28,  Ed.]  should  be  published  and  used  to 
evaluate  any  proposed  scheme  for  Improved  data  types. 

Operating  System  Design  For  Reliability  (Linden) 

Methods  for  taking  an  operating  system  from  description  to  the 
executable  code  were  discussed. 


Neumann's  five  stage  plan  was: 


1)  structural  decomposition  into  hierarchical  virtual  machines, 

2)  establishment  of  assertions  about  the  interfaces  of  successive 
virtual  machines, 

3)  specification  of  algoritnms  for  realizing  one  virtual  machine 
in  terms  of  the  operations  of  lower  level  virtual  machines, 

4)  programming  the  algorithms, 

5)  compiling  the  programmed  procedures. 


be  exploited.  A  good  user  interface  (such  as  in  UNIX)  permits 
to  create  ' arge  systems. 
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Session:  Reading  Programs 
Chairman:  B.  Hetzel 


Recorder:  Schild 


HETZEL  pointed  out  that  we  always  hear  and  we  all  agree:  people 
should  read  programs.  But  we  tend  to  ignore  the  equally  important 
question:  how  should  programs  be  read? 

He  presented  (as  a  question)  the  following  methodology  which  has  been 
used  successfully  in  the  Safeguard  Code  Certification  Experiment: 

(Note:  Well  structured  programs  are  assumed!)  The  outline  is 

1)  Become  familiar  with  a  general  overview 

2)  Read  the  program  bottom  up  (most  nested  level  first). 

3)  Read  the  program  top  down. 

4)  Compare  the  resulting  effect  with  the  specifications. 

For  (1)  the  overview  is  provided  by  the  author  of  the  program. 

For  (2)  the  reader  has  a  listing  of  the  program  divided  up  into  paragraphs 
(segments)  no  longer  than  1/2  page,  together  with  the  symbol  table  listing 
the  identifiers  with  their  attributes.  For  each  paragraph  he  then  completes 
a  Paragraph  Effects  Summary  sheet. 


name 

used 

set 

used 

after 

set 

local 

comments 

Effect 

He  enters  the  identifier  names  himself  (though  that  could  of  course 
be  done  automatically)  and  puts  check  marks  in  the  next  four  columns  as 
he  reads  the  paragrapn,  statement  by  statement.  Loops  are  gone  over 
three  times.  If-statements  are  checked  for  both  values  of  the  condition. 

In  the  comments  space  he  puts  down  whatever  seems  useful,  so  that  finally 
he  can  state  exactly  what  the  paragraph  does:  its  Effect. 

This  is  done  gradually  moving  to  less  and  less  local  levels.  Finally, 
step  (3)  is  performed,  where  the  main  program  is  read  top  down,  i.e. 
in  terms  of  the  paragraphs  that  have  been  analysed.  This  results  in  an 
Effect  for  the  entire  program,  which  then  is  (in  step  4)  compared  to 
the  specifications  (supplied  with  the  program). 
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Discussi on : 


WEISSMAN  wanted  to  know  why  the  reading  was  not  done  top  down.  The 
answer  is  that  there  are  advantages  and  disadvantages  to  either  method, 
and  they  had  chosen  that  one. 

WEISSMAN  stated  that  in  his  expe-'iments  people  resented  being  told 
exactly  how  to  read  a  program,  but  he  did  not  find  any  difference  between 
the  performance  of  subjects  who  could  do  as  they  chose  and  those  who  had 
to  use  a  prescribed  method:  on  the  average  they  all  arrived  at  the  same 
understanding  of  the  program. 

BALlARD  asks  why  the  reading  is  done  at  all.  Answer:  either'  for 
checking  a  new  program  (before  extensive  testing  is  performed,  someone 
other  than  the  author  goes  over  the  p'^ogram  in  the  way  described)  or 
familiarizing  oneself  with  an  existing  program. 
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Session:  Data  Types  and  Reliability  II 
Chairman:  Bill  Wulf 


Recorder:  Horspool 


The  ostensible  purpose  of  the  second  session  was  for  ALPHARD  and  CLU  to  be 
compared  against  the  list  compiled  in  the  first  session  [page  28,  Ed.].  As  things 
turned  out,  Bill  Wulf  began  by  descr-'bing  forms  (the  equivalent  of  clusters 
or  modules)  in  ALPHARD  and  then  having  to  fend  off  repeated  attacks  on  his 
language  design  decisions.  [Incidently ,  he  revealed  that  his  languag  gets 
its  name  from  Alphard  being  the  briglitest  star  in  the  constellation  Hydra.] 

To  motivate  the  issues,  here  is  the  outline  program  which  started 
all  the  arguments: 


form  stack( n: int)  = 
decl 


!'■ 


common 

unique 


s: vec(int,n) ,sp:int; 


The  unique  option  makes  most  sense  for  stacks.*/ 


initu:  sp^O;  initc:  ... 
finalu:  ...  finale:  ... 


ex  fen  push(z:int)=  ... 


The  form  could  then  be  instantiated  by: 


decl 


own 
1  ocal 


Brief  explanation: 


S :stack(l 3) , 


1.  vec  is  another  form  acting  as  a  storage  allocator.  When  instantiated, 
it  returns  a  vector  of  the  specified  length. 

2.  unique  is  an  attribute  specifying  that  each  instance  of  stack  has  its 
own  unique  instances  of  s  ano  sp. 

3.  Conversely,  common  would  specify  that  all  instances  of  stack  would 
share  the  same  instance  of  s  and  sp. 

4.  initu  labels  code  executed  when  each  instance  of  stack  is  created, 
whereas  initc  labels  code  executed  only  for  the  first  instantiation. 

5.  finalu  labels  code  executed  when  an  instance  is  destroyed,  finale 
labels  code  executed  when  the  the  last  instance  of  stack  is  destoyed. 

6.  The  attributes  own  and  local  specify  the  life-time  of  the  object,  S. 
The  own  attribute  has  much  the  same  meaning  as  in  Algol. 

7.  The  ^  attribute  permits  the  push  operation  to  be  visible  outside 
the  form.  (e^E  is  an  abbreviation  for  export.) 

HORNING  argued  that  the  unique  and  common  attributes  indicated  confusion 
between  a  data  type  and  the  function  of  a  storage  allocator.  Neither  did 
Horning  like  the  own  and  local  attributes,  citing  his  experience  that  every 
procedure  containing  own  storage  would  be  better  written  as  a  data  object 
(the  0^  storage)  with  the  procedure  as  an  attribute. 

MITCHELL  and  ZILLES  pursued  the  storage  allocation  topic  with  examples 
of  stacks  obtaining  their  storage  from  one  or  more  pools.  After  much 
debate,  it  was  generally  agreed  that  the  effects  of  the  common/unique 
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attributes  cauld  be  obtained  in  other  waj^s.  However  forms  would  have  to 
be  nested  inside  other  forms,  so  increasing  visual  complexity  of  programs. 

WULF  claimed  that  the  great  virtue  of  the  common  attribute  was  that  common 
storage  would  not  get  instantiated  until  required,  whereas  most  other 
approaches  would  create  the  storage  pool  in  advance. 

WULF  revealed  the  ALPHARD  notation  ior  restricting  the  capabilities 
of  form  instances.  For  example,  when  passing  the  stack  instance,  S,  to 
the  function,  f,  writing  f(S<push>)  would  only  permit  f  to  use  the  push 
operation  on  S .  In  addition,  function  definitions  can  state  a  minimum 
set  of  capabilities.  For  example,  fen  f (z:stack<push  ,pop>)  would  insist 
that  f  be  called  with  at  least  the  capabilities  of  using  push  and  pop 
operations  on  z.  MITCHELL'S  opinion  was  that  it  would  be  preferable  to 
place  capability  restrictions  in  declarations  rather  than  in  function  calls. 

STOY  suggested  the  notation:  form  S'  =  S<push>. 

ROSS  suggested  that  Wulf  was  over-working  scope  rules.  He  distinguished 
four  stages  for  a  form,  each  with  their  cwn  scopes  or  extents.  These  were: 

1 .  Define 

2.  Declare  (specifying  where  the  form  may  be  used) 

3.  Create 

4.  Exist 

But  Ross  confessed  to  not  having  a  suitable  syntactic  notation  for  specifying 
the  various  scopes. 

Finally,  when  time  was  running  short,  the  object  of  the  second  session 
was  achieved.  WULF  quickly  ran  through  the  list  of  session  I  and  made  the 
following  comments: 

1,2:  Visibility  is  explicitly  controlled  with  the  ex  attribute. 

3,4:  Tne  init  and  final  code  provides  explicit  control. 

5:  Can  generate  types  by  passing  forms. 

6:  All  type-checking  is  at  compile-time. 

7:  ALPHARD  dodges  the  issue. 

8:  Problem  not  fixed.  Current  version  is  like  Simula  67. 

9;  Covered  previously. 

10:  Don't  want  this  ability. 

11:  This  is  still  a  problem. 

[The  numbers  indicate  the  corresponding  items  in  the  list  of  session  I,  Page  28.] 
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Session:  Should  a  Professional  School  of  Computation  be  Established? 

Chairman:  Bill  McKeeman  Recorder:  Cherniak 


Among  those  present,  there  was  a  general  agreement  as  to  need:  A 
school  which  concentrates  not  on  research  but  on  developing  people  skilled 
in  the  efficient  production  of  reliable  software  is  very  necessary.  It 
was  recognized  that  not  everyone  is  in  favour  of  such  a  school:  The 
argument  against  usually  centers  aroun  1  the  fear  "Let's  not  make  them  any 
narrower  than  we  do  already". 

Some  first  steps  toward  such  a  school  have  been  taken  at  a  few  places: 
Warsaw  (Tiirski),  Newcas tle-Upon-Tyne  (Brian  Randell),  Johns  Hopkins 
(Harlan  Mills),  Albany  (Weinberg). 

The  cuestion  of  how  such  a  school  would  be  set  up  was  first  addressed 
by  TURSKI.  He  first  described  the  present  system  in  Poland:  The  Informatics 
programme  is  one  of  four  within  Mathematics.  After  the  completion  of  5 
years  (after  8  elementary,  4  secondary)  one  is  awarded  the  M.S.  Intake  to 
the  programme  is  110-120  per  year,  and  about  70%  of  those  complete  it.  The 
programme  is  very  rigidly  structured  (very  few  electives)  and  contains  a 
plateau  after  the  second  year.  The  first  two  years  provide  the  foundations 
in  both  Mathematics  and  Informatics.  The  mathematics  courses  have  an 
Informatics  flavour.  (In  Analysis,  one  concentrates  on  those  things  useful 
for  numerical  math;  in  Algebra,  points  of  special  interest  include  the 
properties  of  semigroups).  Under  Informatics  the  first  semester  course 
provides  a  shallow  overview  while  in  tie  second  semester,  the  student  learns 
about  programming.  During  this  time,  ie  also  learns  something  about  hardware, 
including  structures  and  microstructures  and  even  some  Telecommunications, 
so  that  he  will  be  prepared  to  interact  confidently  with  Electrical  Engineering 
types.  TPe  second  year  contains  two  courses:  Programming  Languages  with  many 
practical  exercises  with  the  computer  -  by  the  end,  each  student  must  program, 
document  and  prove  a  non-trivial  system  (e.g.,  the  8  queens  problem).  The 
Operating  Systems  course  discusses  the  various  components  of  Operating  Systems 
and  concentrates  on  the  related  programming  considerations.  By  the  end  of 
second  year,  students  have  also  studied  some  History,  Philosophy,  etc.  At 
the  beginning  of  third  year  comes  a  split:  One  stream  concentrates  on 
Numerical  Analysis,  the  other,  leading  toward  careers  as  system  programmers 
and  language  designers,  studies  such  topics  as  Compiler  Writing  Techniques, 

Data  Structures  and  Automata  Theory,  with  more  and  more  practical  work  and 
eventually  a  Thesis.  Each  student  must  take  at  least  one  elective  from  "the 
other  side". 

Turski  went  on  to  point  out  the  deficiencies  of  such  a  system,  the 
primary  one  being  the  lack  of  an  exposure  to  pressures  and  the  environment 
which  will  influence  his  programming  in  the  real  world. 

Turski  concluded  by  describing  his  hopes  for  a  solution.  He  envisages 
a  permanent  workshop  for  people  with  very  specific  plans.  He  notes  that 
integration  with  industrial  software  was  a  must.  He  described  his  first 
approach  toward  achieving  the  industrial  environment,  an  assignment 
of  people  to  groups  and  a  3  month  rotation:  one  group  would  start  an 
assignment  and  work  at  it  for  3  months,  then  move  on.  Another  group  would 
take  over  where  the  first  group  left  off,  the  first  group  not  being  graded 
until  the  second  group  finished. 
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As  Turski  sat  down,  he  was  asked  how  the  Polish  refer  to  Polish 
(Lukasiewicz)  notation.  After  giving  the  reply  "parenthesis-free"  he 
went  on  to  describe  the  origin  of  this  notation.  It  seems  that  the 
parentheses  on  Lukasiewicz's  typewriter  were  upper  case  characters, 
and  that  he  was  too  lazy  to  shift. 

McKEEMAN  took  over,  and  described  a  7  year  professional  program, 
a  School  of  Computation  along  the  lines  of  a  School  of  Medicine  with  a 
BC  (Bachelor  of  Computation)  after  4  years,  a  MC  after  five,  and  a  DC 
after  seven.  People  with  a  4  year  BA  in  Computer  Science  [i.e.,  the 
Academic  Stream)  could  enter,  if  they  cesired,  into  the  program  at  the 
beginning  of  fourth  year.  Students  in  the  professional  program  would 
acquire  a  wide  background  -  no  students  would  be  allowed  to  avoid  either 
the  commercial  or  scientific  sides  of  computing.  As  well,  they  would 
gain  such  sundry  skills  as  the  ability  to  examine  the  insides  of  a  computer 
and  determine  which  of  a  set  of  circuit  boards  was  faulty,  or  to  bring 
a  computer  up  cold. 

A  School  of  Computation  would  have  some  advantages  over  a  program 
within  an  academic  environment:  Outside  specialists  (e.g.,  Harlan  Mills) 
could  be  brought  in  easily,  without  creating  appointments;  skills  and 
techniques  courses  would  be  allowed  [Such  courses  are  prohibited  at  UC 
Santa  Cruz,  Ed.]  (in  addition  to  ideas  and  concepts  courses).  The 
possibility  of  a  standard  professional  exam  was  also  mentioned. 

At  this  point  the  session  became  a  wide-open  discussion.  Several 
questions  were  asked  and  suggestions  made,  some  supported,  others 
shot  down : 

SITES:  What  about  the  subject  "Teaching  Computer  Science?"  McKeeman 
noted  that  this  problem  was  not  unique  to  the  field  of  Computer  Science, 
but  offered  that  he  didn't  think  his  professional  school  would  worry  about  it. 

GRAHAfI:  What's  wrong  with  the  present  system:  people  graduate,  get 
hired,  and  then  they're  on  their  own,  and  either  perform  or  get  out!  It 
was  pointed  out  that  usually  they  get  hired  and  stay  on  forever. 

McKEEMAN  pointed  out  that  industry  doesn't  want  graduates  from  the 
research-0 ''iented  universities,  but  from  the  Polytechni cal  Institutes, 
where  they  get  people  who  are  capable  cf  functioning  within  a  few  weeks 
of  starting  in  the  company  -  "people  from  universities  just  won't  work". 

SEVCIK  suggested  an  internship  at  the  end  of  the  training,  or  during, 
as  in  the  co-op  programs  at  Waterloo  and  M.I.T. 

McKEEMAN  argued  that  companies  usually  don't  like  juniors  mucking  up 
the  works.  Other  criticisms  were  levelled  at  this  system  -  e.g., 
students  ai'e  often  given  Joe  jobs  and  derive  little  benefit. 

GRAHAM  thought  it  good  that  people  should  see  what  goes  on  inside 
a  business  the  politics  involved,  the  personality  fights,  everything. 

SEVCIK  noted  that  it  was  only  serving  to  transplant  the  adjustment 
period  from  after  graduation  to  an  earlier  time. 
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HOLT  brought  up  the  point  that  surh  a  programme  keeps  the  schools 
(professors)  honest:  They  must  keep  up-to-date  to  keep  the  students' 
respect. 

SEVCIK  suggested  as  an  alternative  that  the  professional  school 
attract  software  contracts  for  students  to  work  on. 

GRAHAM  thought  this  too  risky. 

TURSKI  believed  that  it  would  be  OK  if  set  up  properly.  An  analogy 
was  made  with  Dental  Schools,  where  trainees  work  on  volunteers  under 
close  supervision.  TURKSI  suggested  that  the  student  not  be  allowed  to 
go  on  until  the  company  was  satisfied  vith  and  paid  for  the  contract. 

GOTLIEB  thought  the  notion  too  difficult  to  defend  when  one  considered 
the  unfair  competition  to  budding  software  development  houses. 

YEH  asked  what  route  should  be  taken  by  a  company  which  seeks  to 
upgrade  its  people,  but  can't  consider  sending  them  through  a  whole 
graduate  aegree. 

McKEEMAN  mentioned  that  summer  courses  have  been  partially 
successful . 

TURSKI  referred  to  an  article  in  IFIP:  Programming  Teaching 
Techniques  -  Working  Conference  at  Zacopani ,  Poland  (1973)  on  how  people 
in  Phillips  kept  people  up-to-date  with  co-operation  of  the  University. 

Practical  skills  were  again  discussed,  TURSKI  extending  his  constant 
analogy  with  medicine:  "Anatomy  of  Programming",  "Pathology  of  Programming". 

McKEEMAN  in  a  discussion  about  reading  program,  said  that  there 
were  very  very  few  large  programs  worth  reading  for  their  good  qual i  ties , 
most  were  examples  of  how  not  to  do  it. 

The  session  ended  with  an  agreement  among  participants  to  keep  in 
touch  with  new  ideas. 
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Appendix  A 


Workshop  Participants 


Alan  Ballard, 

Department  of  Computer  Science, 
University  of  British  Columbia, 
2075  Wesbrook  Place, 

Vancouver,  B.C.,  Canada, 

V6T  1W5. 

David  Barnard, 

Computer  Systems  Research  Group, 
University  of  Toronto, 

Toronto,  Ontario,  Canada, 

M5S  1A4. 

Barry  W.  Boehm, 

Software  Research  &  Technology, 

TRW, 

One  Space  Park, 

Redondo  Beach,  California,  90278, 

U.S.A. 

Thomas  H.  Bredt, 

Digital  Systems  Laboratory, 

Stanford  University, 

Stanford,  California,  94305, 

U.S.A. 

John  R.  Brown, 

TRW  Systems  R2/2412, 

Redondo  Beach,  California,  90278, 

U.S.A. 

Wi 1 1 i am  C .  Carter, 

IBM  Research, 

P.O.  Box  218, 

Yorktown  Heights,  New  York,  10598, 

U.S.A. 

Ri  chard  des Jardi ns  , 

Spacecraft  Control  Programs  Branch, 
Code  514, 

Goddard  Space  Flight  Center, 
Breenbelt,  Maryland,  20771, 

U.S.A. 

David  B.  DeVaney, 

Honeywell  Information  Systems  Inc., 
MS  805A, 

300  Concord  Road, 

Bi 1 leri ca.  Mass .  ,  01821 , 

U.S.A. 

B.N.  Dickman, 

Bell  Laboratories, 

444  Hoes  Lane, 

Piscataway,  N.J.  ,  U.S.A. 


Richard  E.  Fairley, 

Department  of  Computer  Science, 

University  of  Colorado, 

Boulder,  Colorado,  80302, 

U.S.A. 

Peter  Freeman, 

Uni/ersity  of  California,  Irvine:, 

Department  of  Information  &  Computer  Science 
Irvine,  California,  92664, 

U.S.A. 

R.  Stockton  Gaines, 

Communications  Research  Division, 
von  Neumann  Hal  1 , 

Institute  for  Defence  Analysis, 

Princeton,  New  Jersey,  08540, 

U.S.A. 

John  Gannon, 

Department  of  Computer  Science, 

University  of  Toronto, 

Toronto,  Ontario,  Canada, 

MSS  1A7. 

E.  Girard, 

Thomson  -  CSF, 

33  rue  de  Vouille, 

75015  Paris , 

France . 

John  Goodenough, 

Sof Tech , 

460  Totten  Pond  Road, 

Waltham,  Mass . ,  02154, 

U.S.A. 

Robert  Graham, 

Department  of  Computer  Science, 

The  City  College, 

Convent  Avenue  at  138th  Street, 

New  York,  New  York,  10031, 

U.S.A. 

Jarfies  Gray, 

IBM  Research, 

Monterey  &  Cottle  Roads, 

San  Jose,  California,  95193, 

U.S.A. 

Bi 1 1  Hetzel , 

UNC  Computation  Center, 

University  of  North  Carolina, 

Chapel  Hill,  North  Carolina,  27514, 

U.S.A. 
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Janies  Horning, 

Computer  Systems  Research  Group, 

University  of  Toronto, 

Toronto,  Ontario,  Canada, 

MSS  1A4. 

W.  Howden, 

University  of  California,  Irvine, 

Department  of  Information  &  Computer  Science, 
Irvine,  California,  92664, 

U.S.A. 

Brian  W.  Kernighan, 

Bell  Laboratories, 

600  Mountain  Avenue, 

Murray  Hill,  New  Jersey,  07974, 

U.S.A. 

Richard  B.  Kieburtz, 

State  University  of  New  York  at  Stony  Brook, 
Department  of  Computer  Science, 

Stony  Brook,  New  York,  11790, 

U.S.A. 

James  C.  King, 

IBM  Research, 

P.O.  Box  218, 

Yorktown  Heights,  New  York,  10598, 

U.S.A. 

Karl  N.  Levitt, 

Information  Science  Laboratory, 

Stanford  Research  Institute, 

Menlo  Park,  California,  94025, 

U.S.A. 

Theodore  A.  Linden, 

4964  Woodward  Gardens, 

Columbia,  Maryland,  21044, 

U.S.A. 

Barbara  Liskov, 

Massachusetts  Institute  of  Technology, 

Project  MAC, 

545  Technology  Square, 

Cambridge,  Mass.,  02139, 

U.S.A. 

Carlos  J.  Lucena, 

Computer  Science  Department, 

3532  Boel ter  Hal  1 , 

University  of  California, 

Los  Angeles,  California,  90024, 

U.S.A. 


Nico  Lomuto, 

Cable  Network  -  Analysis  Group, 

Bell  Laboratories, 

Whippany  Road, 

Whippany,  New  Jersey,  07981, 

U.S.A. 

Ralph  Londonj 

use  Information  Sciences  Institute, 
4676  Admiralty  Way, 

Marina  del  Rey,  California,  90291, 

U.S.A. 

Willi  am  M.  McKeeman , 

Information  Sciences, 

University  of  California, 

Santa  Cruz,  California,  95064, 

U.S.A. 

Mi ke  Mel liar-Smi th , 

Computing  Laboratory 
The  University  of 
Newcastle-upon-Tyne, 

NEl  7RU, 

England. 

Edward  F.  Mi  1  ler ,  Jr.  , 

General  Research  Corporation, 

5383  Hollister  Avenue, 

P.O.  Box  3587, 

Santa  Barbara,  California,  93105, 

U.S.A. 

James  S.  Miller, 

Intermetrics  Inc., 

701  Concord  Avenue, 

Cambridge,  Mass.,  02138, 

U.S.A. 

James  G.  Mitchell, 

Xerox  Corporation, 

Palo  Alto  Research  Center, 

3180  Porter  Drive, 

Palo  Alto,  California,  94304, 

U.S.A. 

Peter  Neumann, 

Stanford  Research  Institute, 

Menlo  Park,  California,  94025, 
U.S.A.  '!5  J 

Larry  Ragland, 

Department  of  Computer  Science, 

The  University  of  Iowa, 

Iowa  City,  Iowa,  52242  , 

U.S.A. 
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Lawrence  Robinson, 

Computer  Science  Group, 

Stanford  Research  Institute, 

333  Ravenswood  Avenue, 

Building  202B, 

Menlo  Park,  California,  94205, 

U.S.A. 

Raymond  T.  Yeh, 

Department  of  Computer  Sciences, 
Physics  Building  3.28, 

The  University  of  Texas  at  Austin, 
Austi n ,  Texas  ,  78712 , 

U.S.A. 

Douglas  T.  Ross, 

SofTech , 

460  Totten  Pond  Road, 

Waltham,  Mass . ,  02154, 

U.S.A. 

Larry  Weissman, 

Department  of  Computer  Science, 
University  of  Toronto, 

Toronto,  Ontario,  Canada, 

MSS  1A7. 

Martin  Shooman, 

Polytechnic  Institute  of  New  York, 

Long  Island  Center, 

Route  110, 

Farmingdale,  Long  Island,  New  York,  11735, 

U.S.A. 

Stephen  N.  Zilles, 

IBM  SDD  Cambridge  Systems  Group, 
545  Technology  Square, 

Cambridge,  Mass.,  02139 

Mary  Shaw, 

Carnegi e-Mel  Ion  University, 

Department  of  Computer  Science, 

Schenley  Park, 

Pittsburgh,  Pennsylvania,  15213, 

U.S.A. 

Richard  L.  Sites, 

Computer  Science  Department, 

Stanford  Uni/ersity, 

Stanford,  California,  94305, 

U.S.A. 

» 

Joseph  E.  Stoy, 

MIT/Project  MAC, 

545  Technology  Sqaure, 

Room  522, 

Cambridge,  Mass.,  02139, 

U.S.A. 

Wladyslaw  M.  Turski , 

Institute  of  Mathematical  Machines, 
Department  of  Informatics, 

Warsaw  University, 

Krzywickiego  34, 

02-078  Warszawa,  Poland. 

Wi 1 liam  A.  Wulf, 

Carnegie-Mel Ion  University, 

Department  of  Computer  Science, 

Schenley  Park, 

Pittsburgh,  Pennsylvania,  15213, 

U.S.A. 
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